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Introduction 


This  documents  reports  the  accomplishments  of  the  PI,  Dr  Mehdi  Moradi,  supported  by  the 
postdoctoral  training  Award  W81XWH-10-1-0201,  from  April  1,  2010  to  March  31,  2012.  The 
progress  on  the  specific  task  listed  in  the  award  will  be  listed.  Since  this  is  a  training  award,  other 
achievements  of  the  trainee  during  the  award  will  be  enumerated  as  well.  The  technology  developed 
for  vibroelastography  was  described  in  the  annual  report.  We  will  focus  on  progress  towards  specific 
clinical  outcomes  and  achievements  of  the  trainee. 

The  rationale  of  the  proposal  is  that  prostate  tumors  cannot  be  visualized  in  the  ultrasound  B- 
mode  images  used  for  interventional  guidance.  As  a  result,  the  current  methodology  of  transrectal 
ultrasound  (TRUS)  guided  biopsy  is  a  systematic  sampling  approach,  which  can  potentially  miss  or 
under  sample  cancerous  tissue  or  may  incorrectly  stage  the  disease.  Similarly,  for  therapeutic 
interventions  such  as  brachytherapy  and  surgery,  the  lack  of  reliable  radiologic  characterization  of 
the  tissue  forces  the  choice  of  radical  treatment  as  opposed  to  focal  therapy.  The  proposed  research 
aims  to  produce  highly  demanded  imaging  technologies  for  ultrasound-based  prostate  biopsy  and 
treatment  procedures. 


Body 

Progress  towards  the  research  tasks  defined  in  the  statement  of  work: 

Task  1:  Data  collection  and  analysis  to  address  specific  aims  (i)  and  (ii)  in  the  research  narrative:  Data 
collection  for  validation  of  in-vivo  accuracy  of  cancer  detection  and  extracapsular  extension  of 
prostate  cancer  based  on  RF  time  series  and  VE.  This  will  be  performed  prior  to  prostatectomy.  The 
gold  standard  will  be  histopathologic  analysis  of  the  extracted  specimen.  This  serves  specific  aims  1 
and  2  in  the  research  narrative. 

Progress  toward  task  1:  data  collection  was  performed  under  UBC  IRB:  "(H08-02696)  Ultrasound 
studies  for  prostate  imaging  optimization”.  The  protocol  was  designed  to  acquire  ultrasound  RF  data 
and  vibroelastography  from  the  patients  in  the  operating  room,  immediately  before  the  radical 
prostatectomy  surgery.  We  scheduled  pre-operative  multiparametric  MRI  from  the  same  cases  as 
well.  After  the  surgery,  we  collected  whole-mount  pathology  analysis  of  the  extracted  prostate  tissue. 
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Due  to  operational  issues,  the  acquisition  of  the  pathology  slides  was  slow  and  in  some  cases 
impossible.  Table  I  lists  the  data  acquired. 

Table  I:  Number  of  cases  and  data  types  acquired  during  the  study 


Total  number  of  cases 

19 

Cases  with  wholemount  pathology 

10 

Cases  with  VE 

19 

Cases  with  RF  ultrasound 

12 

Cases  with  pre-operative  MRI 

14 

Task  2:  Analysis  to  address  specific  aims  (Hi)  in  the  research  narrative:  Data  collection  for 
validation  of  the  visualization  and  segmentation  of  prostate  with  VE  imaging.  MR  imaging  is  used 
for  control. 

Progress  toward  Task  2:  The  PI  and  the  team  in  UBC  have  made  significant  progress  towards 
improving  the  visualization  and  the  segmentation  of  the  gland.  This  has  resulted  in  publications  that 
we  included  as  Appendices.  The  summary  of  the  methodology  and  results  are  provided  here: 

VE  as  a  tool  to  improve  prostate  visualization:  Vibroelastography  (VE),  an  ultrasound-based 
method  that  creates  images  of  tissue  viscoelasticity  contrast,  was  evaluated  as  an  imaging  modality 
to  visualize  and  segment  the  prostate.  We  performed  a  clinical  study  to  characterize  the  visibility  of 
the  prostate  in  VE  images  and  the  ability  to  detect  the  boundary  of  the  gland.  Measures  for  contrast, 
edge  strength  characterized  by  gradient  and  statistical  intensity  change  at  the  edge,  and  the 
continuity  of  the  edges  were  proposed  and  computed  for  VE  and  B-mode  ultrasound  images. 
Furthermore,  using  MRI  as  the  gold  standard,  we  compared  the  error  in  the  computation  of  the 
volume  of  the  gland  from  VE  and  B-mode  images.  The  results  demonstrated  that  VE  images  are 
superior  to  B-mode  images  in  terms  of  contrast,  with  an  approximately  six  fold  improvement  in 
contrast-to-noise  ratio,  and  in  terms  of  edge  strength,  with  an  approximately  two  fold  improvement 
in  the  gradient  in  the  direction  normal  to  the  edge.  The  computed  volumes  showed  that  the  VE 
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images  provide  an  accurate  3D  visualization  of  the  prostate  with  volume  errors  that  were  slightly 
lower  than  errors  computed  based  on  B-mode  images.  The  total  gland  volume  error  was  8.8  ±  2.5% 
for  VE  vs.  MRI  and  10.3  ±  4.6%  for  B-mode  vs.  MRI,  and  the  total  gland  volume  difference  was  -4.6  ± 
11.1%  for  VE  vs.  MRI  and  -4.1  ±  17.1%  for  B-mode  vs.  MRI,  averaged  over  nine  patients  and  three 
observers.  Our  results  showed  that  viscoelastic  mapping  of  the  prostate  region  using  VE  images  can 
play  an  important  role  in  improving  the  anatomic  visualization  of  the  prostate  and  has  the  potential 
of  becoming  an  integral  component  of  interventional  procedures  such  as  brachytherapy.  Full 
description  of  the  technology  and  the  clinical  study  are  published  in  Medical  Image  Analysis. 

Task  3:  Development  of  feature  extraction  and  classification  approaches  for  real-time  tissue  typing 
for  cancer  detection  (based  on  RF  time  series). 

On  cancer  detection,  due  to  technical  issues  regarding  motion  compensation  from  passive  RF  time 
series,  we  focused  mainly  on  cancer  detection  from  ultrasound  elastography  (which  is  the  active 
version  of  the  time  series).  The  most  recent  results  acquired  from  data  are  outlined  here. 

Transrectal  ultrasound  with  VE  was  performed  intra-operatively,  prior  to  the  prostatectomy 
procedure,  on  patients  diagnosed  with  prostate  cancer.  Transfer  function  images  of  the  prostate, 
showing  the  relative  stiffness  of  the  tissue  within  and  surrounding  the  prostate  were  created.  For 
each  case,  9-13  pathology  slides  extracted  from  the  prostate  at  approximately  4mm  intervals,  with 
cancer  marked,  were  available.  Areas  suspected  for  cancer  were  marked  on  the  VE  images  and  then 
compared  to  the  pathology  results. 

Gleason  scores  for  51  tumors  were  available.  Twenty  of  the  31  tumors  with  Gleason  scores  of 
3+3  (64.5%),  13  of  the  16  tumors  with  Gleason  scores  of  3+4  (81.25%),  both  tumors  with  Gleason 
scores  of  4+3  with  tertiary  5  (100%)  and  both  tumors  with  Gleason  scores  of  4+5  (100%)  were 
detected.  VE  had  an  overall  sensitivity  of  75.5%  for  detecting  prostate  cancer,  with  a  false 
negative  and  a  false  positive  percentage  of  24.4%  and  37.3%  respectively.  The  sensitivity  of  VE 
for  detecting  cancer  with  a  Gleason  score  of  7  and  above  was  85%.  A  region  based  analysis  was  also 
performed.  Some  anterior  lying  tumors  were  detected  in  elastography. 

This  study  showed  that  the  use  of  additional  information  from  VE  has  the  potential  of 
improving  the  detection  of  prostate  cancer.  This  alternative  imaging  method  could  aid  prostate 
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biopsy  by  highlighting  areas  suspicious  for  cancer,  reducing  the  need  for  repeated  biopsy  procedures. 
It  could  also  potentially  be  used  as  augmented  reality  during  robotic-assisted  prostatectomy.  An 
abstract  featuring  these  results  will  be  presented  at  the  67th  Annual  Meeting  of  the  Canadian  Urology 
Association  in  Banff,  Alberta,  Canada  [June  24th,  2012). 

A  machine  learning  framework  for  cancer  detection  from  image  data:  During  the  course  of  this 
award,  the  PI  worked  on  a  classification  framework,  based  on  support  vector  machine  classification, 
to  calculate  cancer  probability  maps  from  image  data.  The  details  are  reported  in  Appendix  I  on 
multiparametric  MRI  data.  The  use  of  the  same  approach  for  US  data  is  the  topic  of  a  manuscript 
currently  in  preparation. 

Task  4:  Developing  automatic  segmentation  methods  for  delineating  VE  images  of  prostate  for  real 
time  gland  visualization  during  prostatectomy. 

Progress  toward  Task  4: 

Prostate  segmentation  in  B-mode  images  is  a  challenging  task  even  when  done  manually  by 
experts.  We  proposed  a  3D  automatic  prostate  segmentation  algorithm  which  made  use  of 
information  from  both  ultrasound  B-mode  and  vibroelastography  data.  We  exploited  the  high 
contrast  to  noise  ratio  of  vibroelastography  images  of  the  prostate,  in  addition  to  the  commonly 
used  B-mode  images,  to  implement  a  2D  Active  Shape  Model  [ASM)-based  segmentation  algorithm 
on  the  midgland  image.  The  prostate  model  was  deformed  by  a  combination  of  two  measures,  the 
gray  level  similarity  and  the  continuity  of  the  prostate  edge  on  both  image  types.  The 
automatically  obtained  midgland  contour  was  then  used  to  initialize  a  3D  segmentation  algorithm 
which  models  the  prostate  as  a  tapered  and  warped  ellipsoid.  Vibroelastography  images  were 
used  in  addition  to  ultrasound  images  to  improve  boundary  detection. 

We  reported  a  Dice  similarity  coefficient  of  0.85±0.10  and  0.84±0.09  comparing  the  2D 
automatic  contours  with  manual  contours  of  two  observers  on  61  images.  For  11  cases,  a  whole 
gland  volume  error  of  10.2±2.2%  and  13.5±4.1%  and  whole  gland  volume  difference  of  -7.2±9.1% 
and  -13.3±12.6%  between  3D  automatic  and  manual  surfaces  of  two  observers  was  obtained.  This 
was  the  first  validated  work  showing  the  fusion  of  B-mode  and  vibroelastography  data  for 
automatic  3D  segmentation  of  the  prostate.  Sample  results  are  illustrated  in  Figure  1.  The 
complete  description  of  the  methodology  is  currently  under  review  in  the  IEEE  Transactions  on 
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Medical  Imaging  [1].  We  have  argued,  in  [1],  that  adding  of  vibroelastography  images  to  the 
conventional  B-mode,  for  improving  prostate  segmentation  is  very  helpful  and  important. 

Task  5:  To  investigate  the  sources  of  tissue  typing  information  in  RF  time  series. 

Our  clinical  in  vivo  study  showed  that  our  RF  time  series  analysis  as  reported  in  [2]  and  [3]  on  ex- 
vivo  tissue,  needs  major  modifications  to  be  adopted  for  in-vivo  work.  This  is  due  to  the  fact  that 
the  methods  put  forth  in  [2,3]  rely  on  the  assumption  that  continuously  received  samples  of  RF 
time  series  originate  from  the  same  physical  location  in  the  tissue.  In  the  in-vivo  case,  breathing 
and  patient  motion  violated  this  assumption.  Therefore,  we  need  to  design  motion  compensation 
techniques  to  overcome  this  limitation.  Since  those  methods  are  outstanding,  we  have  sought 
funding  from  other  sources  to  work  on  them.  Since  the  goals  of  this  award  were  clinical,  our 
research  group  decided  to  separate  Task  5  from  the  Pi’s  responsibilities.  Nevertheless,  the  study  of 
the  sources  of  tissue  typing  information  in  RF  time  series  in  ex-vivo  studies  were  accomplished  in 
our  research  group  by  studying  the  thermal  effects  of  ultrasound.  This  is  now  published  in  [4]. 

Task  6:  Training 

The  training  goal  of  this  award  was  to  prepare  the  PI  to  assume  the  role  of  an  independent 
researcher.  We  list  the  training  activities  of  the  PI  here.  It  is  worth  mentioning  that  the  PI  has 
received  a  preliminary  offer  to  start  working  as  a  grant-tenure  track  assistant  professor  in  a  joint 
position  between  the  Electrical  Engineering  and  Urological  Sciences  Department  of  the  University 
of  British  Columbia. 

As  suggested  in  the  statement  of  work,  the  PI  attended  regular  research  meetings  with 
Professor  Tim  Salcudean  (weekly],  and  contributed  to  the  Medical  Imaging  course  (ECE433) 
offered  by  Dr  Robert  Rohling  in  UBC’s  Department  of  Electrical  and  Computer  Engineering  in  Fall 
term  of  2010  and  2012.  The  PI  also  had  a  research  visit  to  the  Brigham  and  Women’s  Hospital  in 
Boston  where  he  started  a  collaboration  with  Professor  Clare  Tempany  from  Harvard  Medical 
School  on  multimodality  prostate  imaging.  This  resulted  in  two  conference  abstracts  that  are 
listed  in  this  report  as  Appendix  VI  and  XI. 
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Figure  1  -  2D  segmentation  results  shown  on  corresponding  B-mode  (left)  and  vibroelastography 
(right)  mid-gland  images.  The  initial  contour  is  shown  in  dashed  yellow  and  the  final  contour  in 
solid  red.  The  final  3D  segmented  surface  (triangulated  blue)  is  also  compared  to  the  manually 

created  surface  (red)  in  the  lower  image. 

Pi’s  conference  presentations  during  the  course  of  the  award: 

1 )  Poster  Presentation:  Towards  joint  MRI-US  based  tissue  typing  and  guidance  for  prostate 
interventions  in  AMIGO,  4th  NCIGT  and  NIH  Image  Guided  Therapy  Workshop,  Washington 
DC,  October  2011. 

2 )  Workshop  style  presentation:  Deformable  prostate  registration,  3D  ultrasound  to  MRI, 
2012  Winter  Project  Week  of  the  National  Alliance  for  Medical  Image  Computing,  January 
2012,  Salt  Lake  City,  Utah. 

3 )  Poster  Presentation:  Towards  intra-operative  prostate  brachytherapy  dosimetry  based  on 
partial  seed  localization  in  ultrasound  and  registration  to  C-arm  fluoroscopy,  MICCAI  2011, 
Toronto,  Canada,  2011. 

4 )  Poster  Presentation:  Prostate  Cancer  Probability  Estimation  Based  on  DCE-DTI  Features 
and  Support  Vector  Machine  Classification,  ISMRM,  Montreal,  Canada,  May  2011. 
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5 )  Poster  Presentation:  Ultrasound-based  techniques  for  enhancing  diagnostic  and 
therapeutic  prostate  interventions,  IMPaCT  2011,  Orlando,  FL,  March  2011.  [Travel  funded 
by  this  award]. 

6 )  Poster  Presentation:  Brachytherapy  strand  visibility  in  reflected  power  ultrasound  images: 
Comparison  of  EchoStrand  vs.  Regular  Strands  Using  3D  Visibility  Profiles,  IEEE  Ultrasonics 
Symposium,  San  Diego,  CA,  2010. 

7 )  Poster  Presentation:  Under-Determined  Non-Cartesian  MR  Reconstruction  with  Non- 
convex  Sparsity  Promoting  Analysis  Prior,  MICCAI,  Beijing,  China,  2010. 

8 )  Poster  Presentation:  Automatic  Prostate  Segmentation  Using  Fused  Ultrasound  B-Mode 
and  Elastography  Images,  MICCAI,  Beijing,  China,  September  2010.  [Travel  partially  funded 
by  this  award]. 

9 )  Podium  Presentation:  Ultrasound  Image  Segmentation  Based  on  Statistical  Unit-root  Test 
of  B-Scan  Radial  Intensity  Profiles,  The  33rd  Conference  of  the  Canadian  Medical  and 
Biological  Engineering  Society,  Vancouver,  Canada,  2010. 

Invited  Tutorials,  talks 

1)  Invited  Research  Seminar:  "Multiparametric  ultrasound  for  tissue  typing,  results  in 
prostate  cancer  characterization",  MIT  Computer  Science  and  Artificial  Intelligence 
Laboratory,  January  2012. 

2)  Research  seminar:  "Image  based  solutions  for  prostate  interventions:  from 
multiparametric  to  multimodality".  Department  of  Radiology,  Brigham  and  Women’s 
Hospital,  Harvard  Medical  School,  October  2011. 

3)  Research  seminar:  "Image-based  solutions  for  prostate  interventions",  Vanderbilt 
University,  Department  of  Biomedical  Engineering,  November  2010. 

4)  A  three  hour  tutorial  titled:  "Imaging  in  Practice:  Comparing  the  Applications  of  X-ray  CT, 
US,  and  MRI  Modalities”,  The  5th  Canadian  Student  Conference  on  Biomedical  Computing 
and  Engineering,  Waterloo,  ON,  Canada,  May  20,  2010. 
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Key  Research  Accomplishments 


The  main  accomplishment  of  this  postdoctoral  fellowship  is  the  success  of  the  PI  who  will  start  an 
independent  career  as  a  researcher  as  of  May  1st,  2012.  The  PI  had  an  excellent  opportunity  to  be 
trained  in  the  general  field  of  image-guided  prostate  interventions.  The  list  of  technical 
contributions  can  be  summarized  as  follows: 

1)  We  showed  that  ultrasound  vibroelastography  is  a  valuable  technology  in  imaging  the 
prostate  gland.  It  improves  the  visualization  of  the  prostate  gland  and  enhances  the 
segmentation. 

2)  We  combined  B-mode  and  vibroelastography  and  created  a  fully  automatic  method  for 
prostate  segmentation  [1]. 

3)  We  studied  ultrasound  vibroelastography  for  cancer  detection  and  showed  that  VE  has  a 
high  sensitivity  in  cancer  detection.  However,  the  specificity  of  this  imaging  technology  is 
not  sufficient  for  cancer  characterization. 

4)  An  ultrasound-based  methodology  for  visualization  of  radioactive  brachytherapy  seeds  in 
prostate  and  dosimetry  was  designed  and  examined  on  a  small  clinical  dataset  [Appendix 
VII]. 

5)  We  developed  a  machine  learning  framework  for  characterization  of  prostate  tissue  using 
features  extracted  from  medical  images.  We  used  this  methodology  as  reported  in 
[Appendix  I]  with  multiparametric  MRI  data.  The  use  of  this  same  technology  with  VE  and 
B-mode  ultrasound  RF  data  is  the  subject  of  a  manuscript  currently  in  preparation. 

Reportable  outcomes 

Journal  Publications: 

1.  M.  Moradi,  S.  E.  Salcudean,  S.  D.  Chang,  E.  C.  Jones,  Nicholas  Buchan,  Rowan  G.  Casey,  S.  L. 
Goldenberg  and  P.  Kozlowski.  Multiparametric  MRI  Maps  for  Detection  and  Grading  of 
Dominant  Prostate  Tumors.  Magnetic  Resonance  Imaging,  In  Press.  DOI:  0.1002 /jmri.23540, 

2012.  [Appendix  I]. 
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9)  S.  S.  Mahdavi,  M.  Moradi,  W.  J.  Morris  and  S.  E.  Salcudean.  Automatic  Prostate 
Segmentation  Using  Fused  Ultrasound  B-Mode  and  Elastography  Images.  In:  MICCAI2010, 
LNCS  6362,  pp.  76-83,  2010.  [Appendix  IX]. 

10)  [Abstract]  Mehdi  Moradi,  et  al.,  "Ultrasound-based  techniques  for  enhancing  diagnostic 
and  therapeutic  prostate  interventions”,  IMPaCT  2011,  Orlando,  FL,  March  2011. 
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11)  [Abstract]  Mehdi  Moradi,  et  al..  Towards  joint  MRI-US  based  tissue  typing  and  guidance 
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Conclusions 

We  have  successfully  completed  four  of  the  five  tasks  defined  in  the  statement  of  work.  One 
task  (the  study  of  the  sources  of  tissue  typing  information  in  RF  time  series)  was  followed 
within  our  research  group,  with  limited  involvement  from  the  PI  due  to  the  technical 
difficulty  of  acquiring  high  quality  RF  time  series  in  vivo.  The  overall  goal  of  the  project 
which  was  the  development  of  ultrasound-based  technologies  for  image-guided  prostate 
interventions  was  fully  achieved.  We  have  developed  methodologies  for  segmentation  of 
the  gland,  cancer  visualization,  and  brachytherapy  seed  detection  during  this  work.  Eleven 
publications  have  been  resulted  from  this  work  and  two  other  submissions  are  currently  in 
preparation.  The  PI  has  also  met  the  training  goals  through  the  mentoring  of  world  class 
scientists  and  clinicians  including  Professor  Larry  Goldenberg,  Professor  Tim  Salcudean, 
and  Professor  Clare  Tempany. 
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Multiparametric  MRI  Maps  for  Detection  and 
Grading  of  Dominant  Prostate  Tumors 
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Purpose:  To  develop  an  image-based  technique  capable  of 
detection  and  grading  of  prostate  cancer,  which  combines 
features  extracted  from  multiparametric  MRI  into  a  single 
parameter  map  of  cancer  probability. 

Materials  and  Methods:  A  combination  of  features 
extracted  from  diffusion  tensor  MRI  and  dynamic  contrast 
enhanced  MRI  was  used  to  characterize  biopsy  samples 
from  29  patients.  Support  vector  machines  were  used  to 
separate  the  cancerous  samples  from  normal  biopsy  sam¬ 
ples  and  to  compute  a  measure  of  cancer  probability,  pre¬ 
sented  in  the  form  of  a  cancer  colormap.  The  classification 
results  were  compared  with  the  biopsy  results  and  the 
classifier  was  tuned  to  provide  the  largest  area  under  the 
receiver  operating  characteristic  (ROC)  curve.  Based  solely 
on  the  tuning  of  the  classifier  on  the  biopsy  data,  cancer 
colormaps  were  also  created  for  whole-mount  histopathol- 
ogy  slices  from  four  radical  prostatectomy  patients. 

Results:  An  area  under  ROC  curve  of  0.96  was  obtained  on 
the  biopsy  dataset  and  was  validated  by  a  “leave-one-patient- 
out”  procedure.  The  proposed  measure  of  cancer  probability 
shows  a  positive  correlation  with  Gleason  score.  The  cancer 
colormaps  created  for  the  histopathology  patients  do  display 
the  dominant  tumors.  The  colormap  accuracy  increases  with 
measured  tumor  area  and  Gleason  score. 

Conclusion:  Dynamic  contrast  enhanced  imaging  and  dif¬ 


1Department  of  Electrical  and  Computer  Engineering,  University  of 
British  Columbia,  Vancouver,  Canada. 

department  of  Radiology,  University  of  British  Columbia,  Vancouver, 
Canada. 

department  of  Urologic  Sciences,  University  of  British  Columbia, 
Vancouver,  Canada. 

4Vancouver  Prostate  Centre,  Vancouver,  BC,  Canada,  Vancouver, 
Canada. 

department  of  Pathology  and  Laboratory  Medicine,  University  of 
British  Columbia,  Vancouver,  Canada. 

6St  George’s  Medical  Centre,  Christchurch,  New  Zealand. 

7MRI  Research  Center,  University  of  British  Columbia,  Vancouver, 
Canada. 

Contract  grant  sponsor:  U.S.  Army  Medical  Research  and  Materiel 
Command;  Contract  grant  number:  W81XWH- 10- 1-0201;  Contract 
grant  sponsor:  the  Canadian  Institutes  of  Health  Research  (CIHR) . 

*  Address  reprint  requests  to:  M.M.,  Department  of  Electrical  and 
Computer  Engineering,  University  of  British  Columbia,  Vancouver, 
Canada.  E-mail:  moradi@ece.ubc.ca 
Received  March  25,  2011;  Accepted  November  22,  2011. 

DOI  10. 1002/jmri.23540 

View  this  article  online  at  wileyonlinelibrary.com. 

©2012  Wiley  Periodicals,  Inc.  1 


fusion  tensor  imaging,  when  used  within  the  framework 
of  supervised  classification,  can  play  a  role  in  characteriz¬ 
ing  prostate  cancer. 

Key  Words:  Multi-parametric  MRI;  support  vector 
machines;  prostate  cancer;  focal  therapy 
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AS  THE  SECOND  leading  cancer -related  cause  of 
death  among  males  (1),  prostate  cancer  is  the  subject 
of  research  efforts  worldwide.  The  improvements 
achieved  in  the  screening  techniques  for  prostate  can¬ 
cer  have  resulted  in  the  diagnosis  of  a  rising  number 
of  cases,  among  which  lies  a  population  with  low  risk 
and/or  localized  tumors.  Nevertheless,  radical  prosta¬ 
tectomy  remains  the  most  common  treatment  for  the 
disease.  As  a  result,  30%  of  cases  of  radical  prostatec¬ 
tomy  are  performed  on  patients  with  pathologically 
insignificant  cancer  (2),  making  them  subject  to  com¬ 
plications  such  as  sexual,  urinary,  and  bowel  morbidity 
after  the  surgery.  These  complications  are  associated 
with  the  techniques  used  during  surgery.  Even  though 
these  have  improved  significantly  in  the  past  decade 
(3,4),  further  improvements  are  difficult  to  envisage 
without  knowledge  of  the  cancer  location,  which  can 
determine  the  need  for  and  extent  of  the  surgical  posi¬ 
tive  margins  applied.  Furthermore,  focal  treatment  of 
the  disease  with  associated  sparing  of  sensitive  struc¬ 
tures  such  as  the  neuro-vascular  bundles,  are  viable 
options  only  if  accurate  patient- specific  maps  of  cancer 
grade  and  stage  can  be  provided  preoperatively.  In  other 
words,  the  success  of  focal  therapy  relies  on  proper 
patient  selection  and  adequate  characterization  of  the 
tumor’s  location,  extent  and  histology.  The  most  com¬ 
monly  used  method  to  assess  prostate  cancer  is  biopsy 
under  transrectal  ultrasound  (TRUS).  Because  TRUS 
cannot  accurately  image  prostate  cancer,  biopsy  proto¬ 
cols  suffer  from  significant  sampling  errors,  resulting  in 
false  negatives  or  under -sampling  of  major  tumors  (5). 

Magnetic  resonance  imaging  has  a  proven  ability  to 
visualize  the  prostate  anatomy  (6).  However,  the  con¬ 
ventional  T2  weighted  MRI  has  insufficient  specificity 
and  sensitivity  for  diagnosis,  and  specially  grading,  of 
prostate  cancer  even  at  3T  (7-9).  Several  MRI-based 
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techniques  that  provide  physiological  information  about 
tissue  have  been  studied  to  improve  the  grading  of  can¬ 
cer.  Earlier  studies  focused  on  Magnetic  Resonance 
Spectroscopy  (MRS),  which  allows  the  creation  of 
metabolite  maps  in  the  prostate  region  (10).  However, 
the  diagnostic  improvements  due  to  MRS  were  limited 
and  inconclusive  (11).  A  more  recent  development  is  the 
use  of  diffusion,  both  diffusion  weighted  imaging  (DWI) 
and  diffusion  tensor  imaging  (DTI),  which  characterize 
the  de-phasing  of  the  MR  signal  caused  by  molecular 
diffusion.  Prostate  cancer  causes  a  pathological  change 
in  the  tissue  in  which  the  regular  pattern  of  distribution 
of  prostatic  glands  is  disrupted  by  masses  of  malignant 
epithelial  cells  and  glands  that  are  irregularly  distrib¬ 
uted  (12).  The  increased  cellular  density  can  cause  a 
decrease  in  apparent  diffusion  coefficients  (ADC)  in 
DWI,  and  average  diffusivity  (D)  values  in  DTI,  meas¬ 
ured  in  prostate  tumors.  The  structural  changes  are 
also  likely  to  alter  the  molecular  level  diffusion  and  to 
cause  differences  in  fractional  anisotropy  (FA)  between 
regular  and  normal  cells,  intensified  by  the  progress  of 
the  disease.  Diffusion  imaging  is  fast  and  convenient 
without  the  need  for  contrast  agents.  Radiologists  have 
reported  that  ADC  can  help  differentiate  central  gland 
tumors  from  stromal  and  glandular  hyperplasia  (13). 
Studies  in  case  of  DTI,  however,  are  also  inconclusive 
and  report  increased,  decreased,  or  even  unchanged 
values  of  FA  (14-16).  Dynamic  Contrast  Enhanced  MRI 
(DCE  MRI)  has  also  been  studied  for  grading  of  prostate 
cancer.  When  combined  with  a  pharmacokinetic  model, 
DCE  characterizes  the  changes  in  tissue  microvascula¬ 
ture  which  are  linked  to  cancer  growth  (17).  However, 
DCE  prostate  studies  report  moderate  to  weak  correla¬ 
tions  between  cancer  grade  and  Gleason  score  with 
individual  DCE  parameters  (18,19).  A  review  of  the  role 
of  MRI  in  detection  and  staging  of  prostate  cancer  is 
recently  published  (20). 

The  inconclusive  correlation  of  MRI-based  physiologic 
parameters  with  the  progression  of  the  disease  is  partly 
due  to  the  heterogeneous  and  complex  nature  of  both 
the  biological  tissue  and  the  physiologic  process  of  can¬ 
cer  progression.  It  is  difficult,  if  not  impossible,  to 
extract  specific  ranges  for  the  values  of  the  individual 
parameters  matched  to  the  cancer  stage.  In  the  absence 
of  such  statistics,  the  solution  appears  to  be  supervised 
classification  of  the  multi-parametric  MRI  data  to  sepa¬ 
rate  normal  and  cancer  tissue  (21-23).  Therefore,  in 
this  work  a  machine  learning  approach  based  on  sup¬ 
port  vector  machine  classification  (SVM)  is  used. 

The  purpose  of  this  work  is  to  use  machine  learning 
to  develop  an  image  based  technique  capable  of  detec¬ 
tion  and  grading  of  prostate  cancer  by  combining  fea¬ 
tures  extracted  from  multiparametric  MRI  into  a  sin¬ 
gle  parameter  map  of  cancer  probability.  We  also 
study  the  validity  of  this  method  by  comparison  of  the 
probability  maps  with  whole-mount  pathology. 

MATERIALS  AND  METHODS 
Data  Collection  Protocols 

The  data  used  in  this  work  were  obtained  in  two  stud¬ 
ies,  both  reviewed  and  approved  by  the  Clinical 


Research  Ethics  Board  of  our  institution.  The  data 
from  a  biopsy  study  were  used  for  training  the 
machine  learning  algorithm  and  leave-one-patient-out 
evaluation.  A  second  dataset  from  patients  scheduled 
for  radical  prostatectomy,  with  available  whole-mount 
histopathology  analysis  after  surgery,  was  used  for 
further  validation  of  the  trained  classifier. 

Biopsy  Data 

These  data  were  used  to  train  classifiers  and  quanti¬ 
tatively  assess  the  performance  of  the  DTI-DCE 
parameters  for  cancer  detection.  During  2009,  29 
patients  with  a  high  clinical  suspicion  for  prostate  ad¬ 
enocarcinoma  due  to  an  elevated  prostate  specific 
antigen  (PSA)  and/or  palpable  prostatic  nodule,  with 
no  prior  treatment,  were  consecutively  recruited  to 
this  study.  Average  PSA  was  8.5  ng/mL  (range:  0.94- 
15  ng/mL).  Written  consent  was  obtained  from  each 
patient  before  entering  the  study.  To  ensure  that  the 
biopsy  process  or  result  did  not  affect  the  assessment 
of  MRI  as  a  diagnosis  tool,  the  MRI  scans  in  the  origi¬ 
nal  study  were  completed  before  biopsy  for  all 
patients;  the  biopsy  was  done  the  same  day  within  a 
couple  of  hours  following  MRI.  We  use  the  same  data 
here,  with  the  goal  of  training  a  classifier  to  detect 
and  grade  prostate  cancer. 

The  biopsies  were  performed  under  local  anesthetic 
and  the  number  of  biopsies  obtained  from  the  periph¬ 
eral  zone  (PZ)  was  determined  by  prostate  gland  size. 
In  patients  with  a  prostate  gland  of  30  cc  or  less,  eight 
biopsies  (base:  right  and  left;  midgland:  right  lateral, 
left  lateral,  right  medial,  left  medial;  apex:  right  and 
left)  were  taken.  For  prostate  glands  ranging  from  31 
to  60  cc,  10  biopsies  (base:  right  lateral,  left  lateral, 
right  medial,  left  medial;  midgland;  and  apex  biopsies 
as  above)  were  obtained.  For  prostate  glands  greater 
than  60  cc,  12  biopsies  were  obtained  (apex:  right  lat¬ 
eral,  left  lateral,  right  medial,  left  medial;  base;  and 
midgland  biopsies  the  same  as  the  10  biopsy  scheme). 

The  dataset  included  a  total  of  240  negative  biopsy 
cores  and  29  positive  biopsy  cores.  The  positive  cores 
were  from  10  patients.  Detailed  information  about  the 
positive  cases  is  provided  in  Kozlowski  et  al  (23)  and 
Table  1.  In  seven  cases,  the  patients  went  on  to  radical 
prostatectomy  as  their  treatment.  In  these  cases,  the 
Gleason  scores  reported  from  postsurgical  analysis  were 
used  as  the  Gleason  score  of  the  tumor.  The  histology 
was  interpreted  with  assignment  of  the  Gleason  score 
by  several  different  experienced  anatomic  pathologists 
who  practice  general  and  subspecialty  uropathology. 

Prostatectomy  Data 

In  2010,  we  started  a  new  study  in  which  patients 
scheduled  for  radical  prostatectomy  consent  to  an 
MRI  imaging  session  before  the  surgery.  This  is  a  dif¬ 
ferent  population  than  the  biopsy  cases  described 
above.  The  patients  recruited  for  this  study  have  not 
received  any  therapy  before  radical  prostatectomy.  At 
the  time  of  writing,  four  datasets  were  obtained,  pro¬ 
viding  40  cross-sections. 

To  acquire  the  whole-mount  pathology  analysis,  the 
radical  prostatectomy  specimens  were  dissected  and 
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Table  1 

Clinical  Data  for  10  Patients  With  Biopsy  Confirmed  Cancer 


Patient 

Age 

[years] 

PSA 

[ng/mL] 

No.  of  positive 
biopsies  (size  in  mm) 

Biopsy  Gleason 
score 

Prostatectomy 
Gleason  score 

Clinical 

stage 

P003 

62 

5.3 

5  (8,  6,  2,  2,  1) 

3+4 

4+3 

cT2a 

P004 

67 

7.1 

4  (7,  5,  1,  1) 

3+4 

3+4 

cT2b 

P005 

60 

5.4 

2(1.5,  1) 

3+3 

n/a 

n/a 

P007 

58 

7.1 

5  (2,  3,  1,  2,  2) 

4+5 

3+4 

cTIc 

P010 

55 

7.09 

2  (0.5,  4) 

3+3 

n/a 

cTIc 

P015 

72 

12 

3  (10,  1.5,  <1) 

4+4,  4+5 

4+5 

cTIc 

P016 

65 

5.43 

2  (<1,  <1) 

4+3,  3+3 

3+4 

cTIc 

P020 

64 

6.8 

2(1,7) 

3+4 

3+4 

cT2a 

P024 

60 

5.4 

1  (3) 

4+4 

4+3 

cTIc 

P026 

62 

6.4 

1  (not  recorded) 

3+3 

n/a 

cT2a 

histopathologically  examined  in  a  uniform  manner. 
The  external  surfaces  of  the  specimens  were  inked 
and  the  seminal  vesicles  amputated.  The  specimens 
were  dissected  following  a  minimum  of  24-h  fixation 
in  10%  buffered  formalin.  The  apical  and  bladder 
neck  tissue  was  removed,  using  5-mm-thick  layers. 
We  developed  a  device  (24)  to  cut  the  prostate  gland 
in  serial  transverse  cuts  perpendicular  to  the  poste¬ 
rior  capsule,  at  4-mm  intervals,  from  inferior  to  supe¬ 
rior.  This  procedure  allowed  us  to  obtain  reasonably 
good  correspondence  between  the  pathology  slices 
and  the  MR  image  slices.  The  processing  and  paraffin 
embedding  were  carried  out  in  a  routine  manner, 
except  using  oversized  baskets  and  blocks  to  accom¬ 
modate  the  intact  prostate  gland.  Whole-mount  sec¬ 
tions  were  cut  using  a  Lieka  RM2245  whole  body 
rotary  microtome,  and  were  submitted  as  intact  trans¬ 
verse  sections  mounted  on  oversize  glass  slides  for 
hematoxylin  and  eosin  staining. 

MR  Imaging  Protocol 

For  both  the  biopsy  and  the  prostatectomy  studies, 
the  MRI  protocol  was  as  follows.  All  MRI  examinations 
were  performed  on  a  3  Tesla  (T)  MRI  scanner  (Achieva, 
Philips  Healthcare,  Best,  The  Netherlands).  MRI  sig¬ 
nals  were  acquired  with  a  combination  of  an  endorectal 
coil  (Medrad,  Pittsburgh,  PA)  and  a  cardiac  phased- 
array  coil  (Philips  Healthcare,  Best,  The  Netherlands). 
Fast  spin-echo  T2 -weighted  images  (repetition  time 
[TR]  =  1851  ms,  effective  echo  time  [TE]  =  80  ms,  field 
of  view  [FOV]  =14  cm,  slice  thickness  =  4  mm  with  no 
gap,  284  x  225  matrix,  3  averages)  were  acquired  in 
the  axial  and  coronal  planes  to  provide  anatomical 
details  of  the  prostate.  From  this  sequence,  12  axial 
slices  covering  the  entire  gland  were  then  selected  and 
used  for  the  DTI  and  DCE  MRI  scans.  DTI  data  were 
acquired  using  a  diffusion  weighted  single  shot  echo 
planar  imaging  (EPI)  sequence  (TR/TE  =  2100/74  ms, 
FOV  =  24  cm,  slice  thickness  =  4  mm  with  no  gap, 
128  x  115  matrix,  6  noncollinear  gradient  directions, 
b-value  =  0  and  600  s/mm2,  18  averages,  total  acqui¬ 
sition  time  of  8  min;  the  relatively  low  b-value  of 
600  s/mm2  was  chosen  to  ensure  sufficient  SNR  for 
quantitative  measurements  of  DTI  parameters). 

DCE  MRI  was  performed  using  a  three-dimensional 
T1 -weighted  spoiled  gradient  echo  sequence  (TR/TE  = 


3.4/1.06  ms,  flip  angle  =  15°,  FOV  =  24  cm,  256  x 
163  matrix,  2  averages).  Initially,  proton  density  (PD) 
images  (TR/TE  =  50/0.95  ms,  flip  angle  =  4°)  were 
acquired  to  allow  calculation  of  the  contrast  agent 
concentrations  in  the  prostate.  Next,  a  series  of 
75  T1 -weighted  dynamics  were  acquired  before 
(3  images)  and  after  (72  images)  a  bolus  injection  of 
Gd-DTPA  (Magnevist,  Berlex  Canada,  0.1  mmol/kg 
injected  with  a  motorized  power  injector  within  10  s 
at  the  rate  of  2  mL/s,  followed  by  a  20-mL  flush  of 
saline.  This  resulted  in  a  time  resolution  of  10.6  s  per 
12  slices.  Currently,  our  scanner  allows  higher  tempo¬ 
ral  resolution;  however,  we  kept  the  same  protocol  over 
the  duration  of  the  study  for  consistency.  T1  values 
were  calculated  based  on  PD -weighted  and  T1 -weighted 
images  according  to  the  procedure  described  in 
Parker  et  al  (25). 

The  total  time  of  the  MRI  examination  was  approxi¬ 
mately  45  min.  The  DTI  data  were  processed  off-line 
to  calculate  FA  and  average  diffusivity  ((D))  values. 
Diffusion  weighted  images  were  registered  to  the 
nonweighted  b  =  0  image  with  a  mutual  information 
algorithm  before  calculating  the  eigenvalues  of  the  dif¬ 
fusion  tensor  and  generating  maps  of  the  average  dif¬ 
fusivity  (D)  (i.e.,  trace  of  the  diffusion  tensor)  and 
fractional  anisotropy  (FA)  with  the  proprietary  DTI 
processing  toolbox  PRIDE  (Philips  Healthcare,  Best, 
The  Netherlands). 

DCE  MRI  data  were  processed  off-line  with  software 
procedures  developed  in  house  using  Matlab  (Math- 
works,  Natick,  MA)  and  Igor  Pro  (WaveMetrics,  Port¬ 
land,  OR).  Arterial  Input  Functions  (AIFs)  were 
extracted  from  voxels  in  the  external  iliac  or  femoral 
arteries  in  the  central  slice  for  each  patient  (26).  Phar¬ 
macokinetic  parameters;  volume  transfer  constant, 
K trans ,  fractional  volume  of  the  extra-vascular  extra¬ 
cellular  space,  ve,  and  fractional  plasma  volume,  vp, 
were  calculated  by  fitting  the  contrast  agent  concen¬ 
tration  versus  time  curves  to  the  extended  Kety 
model.  Fitting  was  carried  out  in  every  pixel  of  every 
slice  within  a  region  of  interest  (ROI)  encompassing 
the  prostate  gland  to  generate  maps  of  the  pharmaco¬ 
kinetic  parameter  as  described  by  Tofts  et  al  (27). 

T2 -weighted  images  were  used  to  identify  the  pros¬ 
tate  gland,  the  peripheral  zone,  and  other  anatomical 
details  that  helped  with  matching  MRI  with  histology. 
DCE  MRI  and  DTI  images  were  acquired  with  the  slice 
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thickness  of  4  mm,  location,  and  the  same  resolution; 
no  interpolation  was  required.  The  vast  majority  of 
the  glands  were  smaller  than  48  mm  and  typically  8  - 
10  slices  were  enough  to  cover  the  entire  gland.  We 
did  not  have  any  cases  where  the  tumor  was  missed 
because  of  the  insufficient  MRI  coverage. 


Matching  ROIs  from  MRI  to  Biopsy  Cores 

In  the  biopsy  data,  one  ROI  represented  by  one  set  of 
DCE  and  DTI  parameters  was  defined  for  each  biopsy 
core.  To  relate  the  DTI  and  DCE  MRI  parameters 
with  biopsy  results,  we  used  the  consensus  in  the  lit¬ 
erature  on  the  values  of  (D)  and  Ktrans  in  prostate 
tumors.  It  is  widely  reported  that  (D)  is  lower  and 
K trCLns  is  higher  in  the  tumor  than  in  normal  PZ 
(19,28). 

The  following  correspondences  between  MRI  slice 
locations  and  gland  anatomy  was  assumed:  slices  5-8 
corresponded  to  midgland  area,  slices  1-4  (2  or  3-4 
for  smaller  glands)  corresponded  to  the  apex,  and  sli¬ 
ces  9-12  (9-10  or  11  for  smaller  glands)  corresponded 
to  the  base.  The  tumor  identification  in  the  MRI  data 
was  based  on  a  threshold  value  determined  separately 
for  the  DTI  and  DCE  MRI  data.  To  determine  the 
threshold,  first  the  areas  with  low  intensity  on  the  (D) 
maps  were  manually  segmented,  based  on  visual 
inspection,  in  the  areas  corresponding  to  the  positive 
biopsy  locations.  For  example,  left  lateral  midgland 
positive  biopsy  corresponded  to  any  low  (D)  value 
areas,  not  smaller  than  3x3  contiguous  pixels 
(approximately  3  mm  x  3  mm),  in  the  left  lateral 
peripheral  zone  within  slices  5-8.  The  intensity 
threshold  for  the  DTI  data  was  defined  as  the  average 
of  all  ROIs  corresponding  to  positive  biopsies  plus  one 
standard  deviation.  The  addition  of  one  standard  devi¬ 
ation  was  applied  to  minimize  the  bias  caused  by 
manual  segmentation.  Similarly  the  intensity  thresh¬ 
old  for  the  DCE  MRI  data  was  determined  as  the  aver¬ 
age  of  the  all  high  intensity  areas  on  the  Ktrans  maps 
corresponding  to  the  positive  biopsies,  minus  one 
standard  deviation.  Once  the  threshold  values  for  the 
DTI  and  DCE  MRI  data  were  established  across  the 
dataset,  the  MRI  data  were  analyzed  again,  and  all 
the  areas  below  the  intensity  threshold  on  the  (D) 
maps  and  above  the  intensity  threshold  on  the  Ktrans 
maps,  corresponding  to  biopsy  locations,  were  con¬ 
sidered  cancer.  In  the  areas  where  no  low  (D)  or  high 
K^ans  vajues  were  present,  and  also  for  negative 
cores,  the  average  parameter  values  from  the  entire 
area  corresponding  to  the  biopsy  location  (e.g.,  aver¬ 
age  of  the  left  lateral  PZ  in  slices  5-8  corresponded  to 
the  left  lateral  midgland  biopsy  location)  were  calcu¬ 
lated  and  used  as  the  feature  vector  representing  the 
biopsy  core. 

It  should  be  noted  that  the  ROIs  from  DCE  and  DTI 
were  selected  separately.  This  was  to  ensure  that  the 
selection  criteria  from  one  modality  did  not  affect  the 
selection  in  the  other  modality.  Furthermore,  the  two 
datasets  were  not  geometrically  registered.  Therefore, 
matching  was  neither  possible  nor  helpful.  Because 
we  classify  an  area  generally  corresponding  to  a 


biopsy,  perfect  geometric  matching  of  the  two  image 
types  is  unnecessary. 

Reference  Standard  in  the  Prostatectomy  Data 

The  histology  slides  were  examined  and  the  regions  of 
the  prostatic  carcinoma  were  outlined  with  assign¬ 
ment  of  the  Gleason  score  by  an  anatomic  pathologist 
with  over  20  years  of  experience  who  practices  general 
and  subspecialty  uropathology.  The  cancer  probability 
maps  acquired  using  the  SVM-based  measure  of  can¬ 
cer  probability  (see  the  next  section)  were  validated 
based  on  these  histopathology  results.  Our  cutting 
method  and  device  ensured  the  matching  of  two- 
dimensional  MRI  slices  with  pathology  slides.  How¬ 
ever,  the  MRI  slices  and  pathology  slides  were  not 
deformed  to  be  registered  to  each  other. 

Classification 

The  MRI  data  processing  resulted  in  three  DCE 
parameter  maps  (Ktrans,  ve,  and  vp)  and  two  DTI 
parameter  maps  ((D)  and  FA)  per  imaging  plane.  Each 
parameter  map  was  of  size  256  x  256  covering  the 
field  of  view.  In  the  biopsy  data,  each  ROI  constructed 
based  on  a  biopsy  core  was  represented  by  the  five¬ 
dimensional  feature  vector  x  =  [Ktrans,  ve,  vp,  ( D ),  FA] 
consisting  of  values  of  the  DCE  and  DTI  parameters 
averaged  over  the  ROI.  To  generate  the  colormaps,  ev¬ 
ery  pixel  of  the  DTI -DCE  image  was  represented  and 
classified  as  a  feature  vector. 

Support  Vector  Machines 

In  its  most  common  form,  the  SVM  classification 
approach  depends  on  using  a  kernel  function  to  map 
the  input  data  to  a  higher  dimension  space  where  a 
hyperplane  can  be  used  to  separate  the  data  into  dif¬ 
ferent  classes.  The  process  of  training  an  SVM  classi¬ 
fier  is  equivalent  to  finding  the  optimal  hyperplane 
that  minimizes  a  classification  error  measure  on  the 
training  dataset  and  maximizes  the  perpendicular  dis¬ 
tance  between  the  decision  boundary  and  the  closest 
data  points  in  the  classes,  a  quality  that  makes  SVM 
a  maximum  margin  classifier.  These  closest  data 
points  that  define  the  decision  boundary  are  called 
“support  vectors”.  In  a  two -class  case,  if  the  training 
dataset  consists  of  N  feature  vectors  fxj,  . . .,  xN}  with 
class  labels  yte  {1,— 1],  then  the  SVM  training  problem 
is  equivalent  to  finding  w  and  b  such  that: 

-u)ju)+cy=1 8,  [i] 

is  minimized  subject  to 

yt(wT<p(xi)  +  b)  >  1  -  8f  [2] 

where  b  and  w  (the  normal  vector)  define  the  hyper - 
plane,  s*  >  0  are  the  so-called  slack  variables  that 
allow  for  misclassification  of  noisy  and  difficult  data 
points,  and  c  >  0  controls  the  trade-off  between  the 
slack  variable  penalty  and  the  error  margin,  and  4>(.) 
is  the  mapping  function  to  a  higher  dimension.  As 
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shown  in  Cortes  and  Vapnik  (29),  w  can  be  written  as 
a  linear  combination  of  the  transformed  support  vec¬ 
tors.  In  other  words,  w  =  <l>  (■*/)»  where  xj s  are  the 

support  vectors.  This  observation  means  that  the  de¬ 
cision  hyperplane  and  the  optimization  Eq.  [2]  only 
contain  the  mapping  function  in  its  dot  product  for¬ 
mat.  Therefore,  one  does  not  need  an  explicit  expres¬ 
sion  for  4>(.)  and  the  SVM  optimization  and  decision 
hyperplane  are  defined  fully  given  the  kernel  function 
of  form  K[xb  Xj)  =  c|)(xJT.  4>(+))  The  most  common  choice 
for  the  kernel  function,  also  used  in  our  work,  is  a 
Radial  Basis  Function  (RBF)  kernel  defined  as: 

K(xi,Xj)  =  e~yl  l*'“*j||2  [3] 

The  use  of  this  kernel  is  equivalent  to  mapping  the 
data  into  an  infinite  dimensional  Hilbert  space  (29). 
The  RBF  kernel  has  only  one  parameter  (y  >  0)  that 
can  be  adjusted.  This  means  that  the  process  of  tun¬ 
ing  the  SVM  classifier  to  a  specific  problem  is  limited 
to  finding  the  appropriate  values  for  just  two  parame¬ 
ters:  c  and  y.  Once  s*  >  0  are  given,  for  a  given  choice 
of  c  and  y,  w,  and  b  that  solve  Eqs.  [1]  and  [2]  are 
obtained  by  solving  a  quadratic  program  (QP).  The 
details  of  the  solution  can  be  found  in  Fan  et  al  (30) 
and  in  the  publicly  available  C++  implementation  of 
the  SVM  algorithms  known  as  LIBSVM  (http://www. 
csie.ntu.edu.tw/~cjlin/libsvm),  which  is  the  library 
that  we  have  used. 

SVM-based  A  Posteriori  Class  Probabilities 

SVM  is  merely  a  decision  machine: 

If  JfvJ  as  +  b  >  0,  then  the  class  label  for  xn  is 

yn=  1.  In  other  words,  SVM  does  not  provide  a  poste¬ 
riori  class  probability.  To  generate  the  cancer  distri¬ 
bution  probability  maps  and  also  the  receiver  operat¬ 
ing  characteristic  (ROC)  curves,  one  needs  a  posteriori 
probability  of  cancer  of  the  form: 

Pc{Xn)  =  p{yn  =  1  L f(xn)  =  WT(p{xn)  +  b)  [4] 

where  Pc(Xn)  stands  for  probability  of  xn  being  cancer¬ 
ous.  To  extend  SVM  for  probability  estimates,  after 
the  training  step,  we  train  the  parameters  of  an  addi¬ 
tional  sigmoid  function  of  the  form: 

Pc{Xn)=p(yn  =  l\fM)=1+exp{2f{xn)  +  B)  [5] 

to  map  the  values  of  JfvJ  to  a  posteriori  probabilities. 
The  values  of  parameters  A  and  B  are  fitted  using 
maximum  likelihood  estimation  from  the  training  set 
for  which  the  actual  labels  are  known  (31,32).  We 
used  class  probabilities  generated  with  this  method 
for  creating  the  probabilistic  cancer  maps. 

Cross  Validation  and  Tuning  the  Classifier 

To  tune  the  classifier,  with  appropriate  values  of  c 
and  y,  we  exhaustively  searched  the  parameter  space 
1  <  c  <  100,  1  <  y  <  100  with  steps  of  0.1.  For  each 
set  of  parameters,  we  trained  the  SVM  using  data 
from  28  biopsy  patients  and  tested  it  on  the  data  from 


the  29th  patient,  and  repeated  the  process  for  all  29 
datasets.  This  exhaustive  search  could  be  completed 
in  less  than  10  min  on  a  typical  personal  computer. 
Note  that  this  process  was  only  completed  once  to  set 
the  parameters  of  the  classifier.  The  classifier  parame¬ 
ter  set  resulting  in  the  best  classification  perform¬ 
ance,  as  measured  by  the  average  area  under  the 
ROC  curve  for  the  29  tests,  was  chosen. 

The  biopsy  dataset  was  unbalanced,  with  more  nor¬ 
mal  biopsy  cores  than  cancerous  cores  (240  versus 
29).  This  could  cause  a  bias  toward  the  normal  class 
during  the  optimization  step  of  the  SVM.  To  reduce 
the  bias,  we  repeated  the  entire  leave-one-patient-out 
training-testing  cross  validation  100  times,  each  time 
with  a  random  subset  consisting  of  half  of  the  normal 
biopsies,  and  averaged  the  results  to  report  the  ROC 
curve.  One-way  analysis  of  variance  (ANOVA)  was 
used  with  the  outcome  of  the  100  trials  to  compare 
the  performance  of  DCE  features  alone,  DTI  features 
alone,  and  DCE-DTI  combined. 

ROCs  and  Colormaps 

To  generate  the  ROC  curves,  a  posteriori  cancer  prob¬ 
ability  (Pc)  was  used  to  determine  a  decision  boundary 
which  we  call  t.  A  biopsy  core  was  classified  as  can¬ 
cerous  if  Pc  >  t.  The  value  of  t  was  incrementally 
increased  from  0  to  1 ,  with  each  t  value  yielding  one 
point  on  the  ROC  curve  (sensitivity  versus  1 -specific¬ 
ity).  To  create  the  colormaps  used  to  validate  the 
methods  by  comparison  with  the  histopathologic  anal¬ 
ysis  of  whole-mount  slides  from  prostatectomy  cases, 
each  pixel  in  the  entire  surface  of  the  prostate  gland 
in  DCE-DTI  images  was  classified  using  the  SVM 
tuned  and  trained  on  the  biopsy  data.  The  Pc  values 
were  plotted  in  standard  Jet  colormap,  with  hot  colors 
representing  high  cancer  probabilities. 

RESULTS 

The  results  reported  here  are  acquired  using  the  SVM 
classifier  with  parameter  values  set  at  c  =  0.6  and  y  = 
4,  determined  by  the  exhaustive  search  to  improve 
the  area  under  the  ROC  curve. 

ROC  Curves  and  Correlation  With  Gleason 
Score  on  Biopsy  Data 

To  study  the  role  of  DCE  and  DTI  separately,  we 
report  three  sets  of  ROC  curves  acquired  using  only 
the  DCE  features,  [Ktrans,  ve,  vp],  only  the  DTI  features 
[(D),  FA],  and  the  combined  feature  vector,  [Ktrans,  ve, 
vp,  (D) ,  FA] .  The  summary  statistics  for  the  five  parame¬ 
ters  in  the  biopsy  dataset  is  presented  in  Table  2,  sepa¬ 
rately  for  biopsy  normal  and  cancerous  regions  of 
interest. 

The  area  under  ROC  curve  (AUC)  was  0.867,  0.919, 
and  0.956  for  DCE,  DTI,  and  the  combined  DCE-DTI 
features  (Fig.  1).  The  combined  feature  vector  resulted 
in  higher  AUC  than  DCE  (P  =  0.002)  and  DTI  (P  = 
0.01).  With  the  combined  feature  vector,  at  the  deci¬ 
sion  threshold  of  Pc  =  0.5,  three  of  the  29  tumors  were 
misclassified  while  a  specificity  of  91%  was  obtained 
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Table  2 

Summary  Statistics  in  Format  of  Mean  (Standard  Deviation)  for  the  Five  Parameters  Used  in  the  Analysis* 


(D)  (1000“3mm2/s) 

FA 

Ktrans  (min  ’) 

ve 

vp 

Normal 

1.662  (0.317) 

0.207  (0.067) 

0.070  (0.047) 

0.227  (0.115) 

0.017  (0.011) 

Cancer 

1.120  (0.174) 

0.198  (0.051) 

0.148  (0.071) 

0.202  (0.049) 

0.007  (0.007) 

*Values  are  reported  for  the  normal  and  cancer  biopsies. 


(22 1  of  243  normal  biopsy  samples  were  correctly  clas¬ 
sified).  When  Pc=  0.3  was  set  as  the  threshold  for  cancer 
detection,  only  one  tumor  was  misclassified,  while  the 
specificity  of  87%  was  maintained  (211  of  243  normal 
biopsy  samples  were  correctly  classified) . 

We  also  noted  a  positive  correlation  between  the  Pc 
value  and  the  Gleason  score  of  the  tumors.  The  aver¬ 
age  Pc  value  was  0.555  for  tumors  of  grade  3  +  3 
(number  of  tumors=7),  0.778  for  tumors  of  grade  3  + 
4  and  4  +  3  (n  =  19),  and  0.963  for  grade  4  +  5  (n  = 
3).  The  increase  in  Pc  values  was  significant  from 
Grade  6  to  7  [P  =  0.01).  The  small  number  of  tumors 
in  grade  4  +  5  did  not  warrant  an  analysis  of  statisti¬ 
cal  significance.  None  of  the  4  +  5  tumors  were  mis¬ 
classified  in  any  of  the  experiments. 

The  T2 -weighted  image  and  the  cancer  probability 
map  obtained  for  the  mid  slice  MRI  data  of  a  patient 
with  positive  biopsy  cores  in  the  mid -left-lateral  and 
mid-left-medial  regions  of  the  prostate  are  illustrated 
in  Figure  2.  This  was  acquired  by  treating  every  pixel 
of  the  DCE-DTI  image  as  a  feature  vector  and  comput¬ 
ing  Pc.  The  tumor  was  of  grade  4  +  5.  For  this  case, 
the  (D)  and  Ktrans  maps  are  also  strong  indicators  of 
the  existence  of  cancer.  These  maps  are  presented  in 
Figure  3. 

Cancer  Colormaps  Revealing  Significant 
Pathologic  Conditions 

The  trained  classifier  was  used  on  the  second  data  set 
from  patients  who  underwent  radical  prostatectomy 
followed  by  whole-mount  pathology  analysis.  In  three 
of  these  four  patients  we  found  a  “dominant  tumor.” 
This  is  in  agreement  with  previous  studies  that  point 
out  the  existence  of  a  single  dominant  tumor  with  sev¬ 
eral  much  smaller  foci  also  present  in  many  prosta¬ 
tectomy  specimens  (33).  In  a  study  involving  100 
prostatectomy  patients,  it  was  shown  that  the  Gleason 

receiver  operating  characteristic  (ROC) 


Figure  1.  ROC  curves  for  the  biopsy  data,  for  different 
groups  of  features  acquired  by  changing  the  decision  thresh¬ 
old,  Pc,  from  0  to  1.  [Color  figure  can  be  viewed  in  the  online 
issue,  which  is  available  at  wileyonlinelibrary.com.] 


score  of  the  dominant  tumor  was  identical  to  the  pri¬ 
mary  Gleason  score  of  the  prostatectomy  specimens 
in  97%  of  patients  (34).  Therefore,  we  believe  a  system 
designed  to  efficiently  screen  patients  for  possible 
focal  therapy,  can  be  assessed  based  on  its  ability  to 
demonstrate  the  dominant  tumor.  Figure  4  shows  the 
outcomes  for  case  1.  For  one  of  the  prostate  sections 
in  this  case,  the  parametric  maps  ( Ktrans  and  (D))  are 
presented  in  Figure  5.  For  case  2  and  3,  samples  of 
the  probabilistic  maps,  along  with  pathology  finding 
and  T2 -weighted  image  are  presented  in  Figures  6 
and  7.  As  these  Figures  illustrate,  in  the  cases  with 
dominant  tumors,  the  cancer  colormaps  created  by 
pixel-wise  computation  of  the  Pc  values  demonstrate  a 
visible  tumor  in  the  area  validated  by  the  histopatho¬ 
logic  analysis.  Note  that  the  DTI  and  DCE  data  from 
these  patients  were  not  used  in  training  the  SVM 
classifier. 

Table  3  reports  the  surface  area  and  the  Gleason 
score  of  the  tumors  determined  by  the  pathologist  and 
compares  them  with  the  surface  area  of  the  tumor 
and  Pc  determined  from  the  colormaps.  To  calculate 
the  surface  area,  pixels  with  Pc  >  0.5  were  considered 
part  of  the  tumor.  For  each  case,  the  data  are  pre¬ 
sented  for  two  consecutive  cross-sections  (pathology 
levels)  that  best  depict  the  dominant  tumor.  In  cases 
with  Gleason  score  >3  +  3,  the  area  of  the  tumor  is 
very  close  in  the  pathology  slide  to  the  area  depicted 
by  the  SVM-based  cancer  colormap  resulting  in  an  av¬ 
erage  difference  of  2.8%.  In  case  of  the  3  +  3  tumor, 
level  4  in  case  2,  the  area  of  the  tumor  as  depicted  in 
the  probability  map  is  40%  smaller  than  the  pathology 
finding.  As  expected,  the  colormaps  more  accurately 
depict  the  tumor  in  the  cases  of  tumor  grades  3  +  4 
and  4  +  5  than  in  tumor  grade  3  +  3.  In  one  case 
(level  10,  case  3),  a  4  +  5  tumor  was  identified.  The 
average  Pc  in  this  case  was  0.86,  providing  a  distinct 
appearance  to  the  tumor.  Note  that  if  one  considers 
the  effect  of  shrinking  due  to  specimen  fixation,  the 
depicted  tumor  area  in  the  colormaps  is  smaller  than 
the  expected  size  of  the  tumor  even  in  case  of  3  +  4 
and  4  +  5  tumors. 

In  one  case,  case  4,  no  dominant  tumor  could  be 
identified.  Cancer  was  bilateral  involving  all  areas  of 
the  gland,  occupying  25%  of  the  overall  volume  of  the 
prostate  and  up  to  60%  of  the  surface  of  certain 
cross-sections.  The  highest  grade  reported  by  the  pa¬ 
thologist  was  3  +  3.  The  colormap  showed  many  hot 
spots.  However,  no  dominant  tumor  was  identified.  In 
the  middle  plane  of  this  3  +  3  case,  the  cancer  area 
depicted  on  the  probability  map,  with  threshold  of 
Pc  >  0.5,  was  36%  smaller  than  the  area  depicted  by 
the  pathologist. 

DCE  imaging  is  not  always  possible  because  con¬ 
trast  agents  are  contra-indicated  in  some  cases.  It 
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Figure  2.  Left:  T2 -weighted  MRI  of  the  mid-gland  of  a  patient  with  biopsy  confirmed  cancer  in  mid-left  region.  Right:  The 
SVM-based  cancer  probability  map,  with  hot  colors  showing  higher  Pc.  The  Gleason  score:  4  +  5,  the  average  Pc  in  the  tumor 
area:  0.9.  [Color  figure  can  be  viewed  in  the  online  issue,  which  is  available  at  wileyonlinelibraiy.com.] 


was  noted  that  DTI  alone  can  also  provide  an  efficient 
modality  within  our  framework:  for  the  three  cases 
with  a  dominant  tumor,  using  only  DTI  features  we 
acquired  maps  showing  an  area  difference  with  the 
combined  DTI-DCE  of  approximately  6.1%  in  the 
dominant  tumor. 


DISCUSSION 

We  found  that  the  combination  of  DTI  and  DCE  pro¬ 
vides  an  area  under  ROC  curve  of  0.96  and  is  more 
efficient  in  cancer  detection  than  each  method  indi¬ 
vidually.  DTI  alone  provides  an  area  under  ROC  curve 
of  0.92  in  the  biopsy  data.  Using  an  extension  of  the 
standard  SVM  classification  approach,  we  were  able 
to  compute  a  posteriori  class  probabilities  that  quan¬ 
tify  the  probability  of  cancer  for  each  set  of  DCE-DTI 
parameters,  based  on  the  available  training  data.  The 
average  Pc  value  was  0.555  for  tumors  of  grade  3  +  3, 
0.778  for  tumors  of  grade  3  +  4  and  4  +  3,  and  0.963 
for  grade  4  +  5. 

One  distinction  of  our  work  compared  with  the  pre¬ 
viously  published  (21,23)  work  on  multiparametric 


MRI  for  cancer  detection  is  our  ability  to  combine  DTI 
and  DCE  features  and  provide  one  reliable  measure  of 
cancer  probability  (Pc)  that  can  be  used  for  ROC 
curves  and  cancer  probability  maps.  We  also  validate 
our  cancer  probability  measure  based  on  the  refer¬ 
ence  standard  of  whole-mount  pathology  on  a  second 
group  of  patients. 

In  the  past,  we  have  reported  the  performance  of 
the  five  parameters  in  detection  of  prostate  cancer 
individually,  and  the  use  of  a  statistical  approach, 
namely  logistic  regression,  for  combining  the  DTI  and 
DCE  parameters  for  cancer  detection  (23).  The  use  of 
SVM  in  the  current  work  provides  two  main  advan¬ 
tages.  The  first  one  is  the  fact  that  SVM  is  a  maximum 
margin  classifier  and  provides  the  ability  to  generalize 
the  trained  classifier.  The  second  one  is  the  ability  of 
the  proposed  framework  to  provide  a  value  for  the 
probability  of  cancer. 

In  analyzing  the  presented  colormaps,  one  should 
note  that  the  training  of  the  SVMs  used  to  compute 
these  probability  values  was  entirely  based  on  the 
data  from  our  biopsy  datasets.  This  means  that 
the  outcomes  are  most  reliable  on  the  peripheral 
zone  of  the  gland,  which  is  usually  the  target  of  the 


Figure  3.  The  (D)  and  Ktrans  maps  for  the  patient  presented  in  Figure  2. 
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Figure  4.  Cancer  colormaps  and  MRI  images  with  corresponding  histopathology  slide  for  case  1 .  The  main  pathologic  finding 
is  a  3+4  tumor  in  the  transition  zone,  visible  in  two  consecutive  cross-sections.  [Color  figure  can  be  viewed  in  the  online 
issue,  which  is  available  at  wileyonlinelibraiy.com.] 


biopsy  studies.  Nevertheless,  as  visible  in  Figure  4, 
we  have  also  been  successful  in  detection  of  the 
dominant  tumor  in  the  transition  zone.  Additional 
data  from  the  transition  zone  are  required  as  we  have 
also  encountered  false  positive  findings  in  this  zone 
(Fig.  6). 

At  this  stage  of  our  work,  we  decided  not  to  use  the 
data  from  prostatectomy  patients  for  the  training  step 
of  the  SVM  classification.  This  was  due  to  several  rea¬ 
sons,  including  the  currently  limited  number  of 
patients  enrolled  in  the  prostatectomy  study,  and  the 
difficulty  of  accurately  registering  the  pathology  slides 
to  MRI  images  to  establish  the  reference  standard  for 
training.  Also,  our  analysis  does  not  target  accurate 


delineation  of  the  individual  tumor  contours  at  this 
stage.  This  is  partly  due  to  the  fact  that  specimen 
deformation  during  fixation  limits  our  ability  for 
establishing  a  reference  standard  for  the  exact  geome¬ 
try  of  the  tumor. 

As  described  earlier,  we  used  a  special  cutting 
device  that  allowed  us  to  slice  the  prostate  in  corre¬ 
spondence  with  the  position  of  the  MRI  slices.  Fur¬ 
thermore,  we  ensured  the  matching  of  the  slices,  visu¬ 
ally,  based  on  the  anatomical  details  clearly  visible  in 
T2 -weighted  images  and  histology  sections.  We 
acknowledge  that  deformations  caused  by  the  fixation 
process  and  the  potential  slight  shift  of  the  slices  in 
the  axial  direction  exist  and  result  in  errors  in  the 
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Figure  5.  The  (D)  and  Ktrans 
maps  corresponding  to  the 
case  presented  in  Figure  4, 
row  1. 
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Figure  6.  Cancer  colormaps  and  MRI  images  with  corresponding  histopathology  slide  for  case  2.  The  main  pathologic  finding 
is  a  tumor  that  is  of  Gleason  score  3  +  4  in  one  cross-section  and  3  +  3  in  the  neighbor  cross-section.  The  hot-spot  in  the 
transition  zone  of  the  prostate  is  a  false  positive  detection.  This  is  most  likely  due  to  the  fact  that  our  SVM  is  trained  only  on 
data  from  peripheral  zone.  [Color  figure  can  be  viewed  in  the  online  issue,  which  is  available  at  wileyonlinelibraiy.com.] 


process  of  matching  the  MRI  and  histology  reference. 
Investigators  have  reported  attempts  directed  toward 
developing  deformable  registration  of  the  histology 
and  MRI  images  (35-38).  However,  the  validation  of 
the  registration  process  requires  maintaining  visible 
markers  throughout  the  imaging,  surgery  and  labora¬ 
tory  work.  This  has  led  to  the  lack  of  a  clinically  viable 
method  for  quantitative  registration  of  MR  slices  with 
histology.  Due  to  this  limitation,  we  have  not  used  the 
data  from  prostatectomy  patients  in  the  training 
stage.  Instead,  we  used  these  data  as  a  second  valida¬ 
tion  dataset.  We  looked  for  the  existence  of  dominant 
tumors  in  the  areas  of  the  slides  corresponding  to 
areas  of  the  cancer  maps  with  high  probability  of  can¬ 
cer.  We  report  the  surface  area  and  the  average  proba¬ 
bility  of  cancer  from  the  maps  and  study  their  correla¬ 
tion  with  the  surface  area  and  Gleason  score  of  the 
tumor  determined  by  the  pathologist.  In  the  absence 
of  a  method  for  quantitatively  validated  deformable 
registration  of  slices  from  MRI  and  histopathology,  we 
have  used  these  measures  for  qualitative  validation. 
In  our  current  setup,  when  most  areas  of  the  gland 
are  invaded  by  carcinoma  (such  as  case  4  in  our  pros¬ 
tatectomy  study),  the  interpretation  and  validation  of 
the  colormaps  is  difficult  due  to  the  lack  of  quantita¬ 
tive  registration.  These  cases,  however,  are  unlikely  to 
be  missed  during  conventional  biopsy.  Furthermore, 
they  are  not  candidates  for  focal  therapy. 


Similar  to  any  other  machine  learning  method,  our 
algorithm  can  be  improved  by  presenting  more  data  in 
the  training  stage.  We  require  more  data  to  represent  all 
levels  of  cancer  progression,  including  very  high  grade 
and  very  low  grade  cases,  and  also  all  areas  of  the 
gland  including  the  central  zone.  As  we  continue  our 
study,  such  data  will  be  incorporated  into  the  system. 
We  will  also  consider  the  detection  of  benign  pathologies 
such  as  benign  prostatic  hyperplasia  when  such  cases 
are  represented  in  our  data  for  training  purposes. 

We  found  that  tumor  classification  is  increasingly 
more  accurate  for  higher  Gleason  scores  (with  2-3% 
area  difference  in  3  +  4  and  4  +  5  tumors  versus  30- 
40%  difference  in  3  +  3  tumors).  The  sensitivity  of  our 
approach  to  the  pathologically  significant  cases,  accu¬ 
racy  in  determining  the  dominant  tumors  with  high 
Gleason  score,  and  the  correlation  of  the  computed 
cancer  probabilities  with  the  Gleason  score,  are  help¬ 
ful  features  for  treatment  planning.  This  provides 
additional  value  to  the  routine  radiological  analysis  of 
the  images,  even  in  cases  where  the  tumor  is  visible 
on  the  T2 -weighted  image. 

As  it  is  evident  from  Figures  3  and  5,  (D)  and  Ktrans 
provide  tumor  contrast  in  many  cases.  Nevertheless, 
clinical  challenges  remain  in  dealing  with  false  posi¬ 
tive  detections,  and  more  importantly,  separating  in¬ 
dolent  from  aggressive  tumors.  This  type  of  prognos¬ 
tic  information  is  currently  available  only  through 
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Figure  7.  Cancer  colormaps  and  MRI  images  with  corresponding  histopathology  slide  for  case  3.  The  main  pathologic  finding 
is  a  tumor  that  is  of  Gleason  score  4  +  3  (with  considerable  tertiary  Gleason  score  of  5)  in  one  cross-section  and  4  +  5  in  the 
neighboring  cross-section. 


pathology,  which  can  be  misleading  due  to  under - 
sampling  of  the  clinically  significant  diseased  areas 
during  biopsy.  Our  computational  framework  is  a 
step  toward  developing  the  ability  to  quantitatively 
characterize  prostate  tumors  noninvasively.  The  clin¬ 
ical  impact  of  such  system,  if  achieved,  will  be  signif¬ 
icant.  We  propose  an  objective,  quantitative  method 
to  radiologically  characterize  prostate  cancer  and 
replace  the  standard  practice  of  visual  evaluation  by 
a  radiologist. 

We  recognize  several  limitations  in  the  current  stage 
of  the  study.  One  limitation  is  the  current  lack  of  a 

Table  3 

Area  and  Gleason  Score  of  the  Tumors  in  Six  Cross-Sections 
From  the  Prostatectomy  Cases  Where  a  Dominant  Tumor  Was 
Present* 


Patient 

Tumor  area 
(pathology) 

Gleason 

Score 

Tumor 
area  (map) 

Pc 

Case  1 ,  level  4 

108  mm2 

3+4 

106  mm2 

0.72 

Case  1 ,  level  3 

94  mm2 

3+4 

92  mm2 

0.70 

Case  2,  level  4 

120  mm2 

3+3 

70  mm2 

0.68 

Case  2,  level  3 

93  mm2 

3+4 

88  mm2 

0.75 

Case  3,  level  9 

86  mm2 

4+3 

83  mm2 

0.82 

Case  3,  level  10 

72  mm2 

4+5 

71  mm2 

0.86 

*For  each  tumor,  the  area  of  tumor  in  the  colormaps  and  also  the 
average  Pc  are  given. 


quantitative  registration  mechanism  between  MRI  and 
pathology.  This  has  limited  our  quantitative  analysis 
on  the  whole-mount  data  to  cases  with  a  dominant 
tumor.  As  described  earlier,  tackling  this  issue  is 
among  our  immediate  research  goals.  The  potential 
geometric  distortion  of  diffusion  images  is  another 
shortcoming.  This  limits  our  ability  to  pinpoint  the 
exact  location  of  a  tumor.  At  the  current  stage,  we 
have  not  detected  significant  distortion  away  from  the 
rectum  in  the  DTI  images  from  the  prostatectomy 
cases  for  which  we  have  whole-mount  pathology.  We 
will  work  toward  a  reliable  technique  to  register  DTI 
data  to  other  MRI  images  as  we  progress.  Finally,  our 
current  validation  with  whole-mount  pathology  is  lim¬ 
ited  to  a  small  dataset  and  we  need  to  expand  this 
study  to  a  sufficiently  large  group  of  patients  who  rep¬ 
resent  different  stages  of  the  disease. 

In  conclusion,  a  combination  of  DCE  and  DTI  MRI 
features,  when  used  along  with  a  SVM -based  machine 
learning  approach,  can  provide  helpful  cancer  proba¬ 
bility  maps  for  detection  and  grading  of  prostate  can¬ 
cer.  The  performance  of  the  method  is  reported  on  a 
biopsy  dataset  of  29  patients,  and  validated  for  three 
additional  cases  by  comparison  of  the  generated  prob¬ 
ability  maps  with  whole-mount  pathology  in  terms  of 
the  accuracy  in  depiction  of  the  dominant  prostate 
tumors. 


Multiparametric  MRI  Maps  for  Prostate  Tumors 


11 


REFERENCES 

1.  Jemal  A,  Bray  F,  Center  MM,  Ferlay  J,  Ward  E,  Forman  D.  Global 
cancer  statistics.  CA  Cancer  J  Clin  2011;61:69-90. 

2.  Steyerberg  EW,  Roobol  MJ,  Kattan  MW,  van  der  Kwast  TH,  de 
Koning  HJ,  Schrder  FH.  Prediction  of  indolent  prostate  cancer: 
validation  and  updating  of  a  prognostic  nomogram.  J  Urol  2007; 
177:107-112. 

3.  Jayram  G,  Decastro  GJ,  Large  MC,  et  al.  Robotic  radical  prostatec¬ 
tomy  in  patients  with  high-risk  disease:  a  review  of  short-term  out¬ 
comes  from  a  high-volume  center.  J  Endourol  2011;25:455-457. 

4.  Shikanov  SA,  Zorn  KC,  Zagaja  GP,  Shalhav  AL.  Trifecta  outcomes 
after  robotic-assisted  laparoscopic  prostatectomy.  Urology  2009; 
74:619-623. 

5.  Odowd  GJ,  Miller  MC,  Orozco  R,  Veltri  RW.  Analysis  of  repeat  bi¬ 
opsy  results  within  1  year  after  a  noncancer  diagnosis.  Urology 
2000;55:533-559. 

6.  Bloch  BN,  Rofsky  NM,  Baroni  RH,  Marquis  RP,  Pedrosa  I,  Lenkin- 
ski  RE.  3  Tesla  magnetic  resonance  imaging  of  the  prostate  with 
combined  pelvic  phased-array  and  endorectal  coils:  initial  experi¬ 
ence.  Acad  Radiol  2004;  1 1:863-867. 

7.  Augustin  H,  Fritz  GA,  Ehammer  T,  Auprich  M,  Pummer  K.  Accu¬ 
racy  of  3-Tesla  magnetic  resonance  imaging  for  the  staging  of 
prostate  cancer  in  comparison  to  the  Partin  tables.  Acta  Radiol 
2009;50:562-569. 

8.  Futterer  JJ,  Heijmink  SW,  Scheenen  TW,  et  al.  Prostate  cancer: 
local  staging  at  3-T  endorectal  MR  imaging-early  experience. 
Radiology  2006;238:184-191. 

9.  Lv  D,  Guo  X,  Wang  X,  Zhang  J,  Fang  J.  Computerized  characteri¬ 
zation  of  prostate  cancer  by  fractal  analysis  in  MR  images. 
J  Magn  Reson  Imaging  2009;30:161-168. 

10.  Kurhanewicz  J,  Vigneron  DB,  Hricak  H,  Narayan  P,  Carroll  P, 
Nelson  SJ.  Three-dimensional  H-l  MR  spectroscopic  imaging  of 
the  in  situ  human  prostate  with  high  (0.24-0. 7-cm3)  spatial  reso¬ 
lution.  Radiology  1996;198:795-805. 

1 1 .  Zakian  KL,  Sircar  K,  Hricak  H,  et  al.  Correlation  of  proton  MR  spectro¬ 
scopic  imaging  with  Gleason  score  based  on  step-section  pathologic 
analysis  after  radical  prostatectomy.  Radiology  2005;234:804-814. 

12.  Issa  B.  In  vivo  measurement  of  the  apparent  diffusion  coefficient 
in  normal  and  malignant  prostatic  tissues  using  echo-planar 
imaging.  J  Magn  Reson  Imaging  2002;16:196-200. 

13.  Oto  A,  Kayhan  A,  Jiang  Y,  et  al.  Prostate  cancer:  differentiation  of 
central  gland  cancer  from  benign  prostatic  hyperplasia  by  using 
diffusion-weighted  and  dynamic  contrast-enhanced  MR  imaging. 
Radiology  2010;257:715-723. 

14.  Gibbs  P,  Pickles  MD,  Turnbull  LW.  Diffusion  imaging  of  the  pros¬ 
tate  at  3.0  Tesla.  Invest  Radiol  2006;41:185-188. 

15.  Manenti  G,  Carlani  M,  Mancino  S,  et  al.  Diffusion  tensor  mag¬ 
netic  resonance  imaging  of  prostate  cancer.  Invest  Radiol  2007; 
42:412-419. 

16.  Xu  J,  Humphrey  PA,  Kibel  AS,  et  al.  Magnetic  resonance  diffu¬ 
sion  characteristics  of  histologically  defined  prostate  cancer  in 
humans.  Magn  Reson  Med  2009;61:842-850. 

17.  Buckley  DL,  Roberts  C,  Parker  GJM,  Logue  JP,  Hutchinson  CE. 
Prostate  cancer:  evaluation  of  vascular  characteristics  with 
dynamic  contrast-enhanced  T1 -weighted  MR  imaging-initial 
experience.  Radiology  2004;233:709-715. 

18.  Engelbrecht  MR,  Huisman  HJ,  Laheij  RJF,  et  al.  Discrimination 
of  prostate  cancer  from  normal  peripheral  zone  and  central  gland 
tissue  by  using  dynamic  contrast-enhanced  MR  imaging.  Radiol¬ 
ogy  2003;229:248-254. 

19.  Padhani  AR,  Gapinski  CJ,  Macvicar  DA,  et  al.  Dynamic  contrast 
enhanced  MRI  of  prostate  cancer:  correlation  with  morphology 


and  tumour  stage,  histological  grade  and  PSA.  Clin  Radiol  2000; 
55:99-109. 

20.  Hoeks  CM,  Barentsz  JO,  Hambrock  T,  et  al.  Prostate  cancer:  mul¬ 
tiparametric  MR  imaging  for  detection,  localization,  and  staging. 
Radiology  201 1;26 1:46-66. 

2 1 .  Chan  I,  Wells  W,  Mulkem  RV,  et  al.  Detection  of  prostate  cancer  by 
integration  of  line-scan  diffusion,  T2-mapping  and  T2-weighted 
magnetic  resonance  imaging:  a  multichannel  statistical  classifier. 
Med  Phys  2003;30:2390-2398. 

22.  Ozer  S,  Langer  DL,  Liu  X,  et  al.  Supervised  and  unsupervised 
methods  for  prostate  cancer  segmentation  with  multispectral 
MRI.  Med  Phys  2010;37:1873-1883. 

23.  Kozlowski  P,  Chang  SD,  Meng  R,  et  al.  Combined  prostate  diffu¬ 
sion  tensor  imaging  and  dynamic  contrast  enhanced  MRI  at  3T- 
quantitative  correlation  with  biopsy.  Magn  Reson  Imaging  2010; 
28:621-628. 

24.  Drew  B,  Jones  EC,  Reinsberg  SA,  Yung  AC,  Goldenberg  SL,  Koz¬ 
lowski  P.  Device  for  sectioning  prostatectomy  specimens  to  facili¬ 
tate  comparison  between  histology  and  in  vivo  MRI.  J  Magn 
Reson  Imaging  2010;32:992-996. 

25.  Parker  GJ,  Suckling  J,  Tanner  SF,  et  al.  Probing  tumor  microvas¬ 
cularity  by  measurement,  analysis  and  display  of  contrast  agent 
uptake  kinetics.  J  Magn  Reson  Imaging  1997;7:564-574. 

26.  Parker  GJ,  Roberts  C,  Macdonald  A,  et  al.  Experimentally-derived 
functional  form  for  a  population-averaged  high-temporal-resolution 
arterial  input  function  for  dynamic  contrast-enhanced  MRI.  Magn 
Reson  Med  2006;56:993-1000. 

27.  Tofts  PS,  Brix  G,  Buckley  DL,  et  al.  Estimating  kinetic  parame¬ 
ters  from  dynamic  contrast-enhanced  T1 -weighted  MRI  of  a  dif- 
fusable  tracer:  standardized  quantities  and  symbols.  J  Magn 
Reson  Imaging  1999;10:223-232. 

28.  Hosseinzadeh  K,  Schwarz  SD.  Endorectal  diffusion-weighted 
imaging  in  prostate  cancer  to  differentiate  malignant  and  benign 
peripheral  zone  tissue.  J  Magn  Reson  Imaging  2004;20:654-661. 

29.  Cortes  C,  Vapnik  V.  Support-vector  networks.  Mach  Learn  1995; 
20:273-297. 

30.  Fan  RE,  Chen  PH,  Lin  CJ.  Working  set  selection  using  the  second 
order  information  for  training  SVM.  Mach  Learn  Res  2005;6: 
1889-1918. 

31.  Platt  JC.  Probabilistic  outputs  for  support  vector  machines  and 
comparison  to  regularized  likelihood  methods.  In:  Smola  AJ, 
Bartlett  PJ,  editors.  Advances  in  large  margin  classifier,  1st  edi¬ 
tion.  Cambridge,  MA:  MIT  Press;  2000.  p  61-74. 

32.  Wu  TF,  Lin  CJ,  Weng  RC.  Probability  estimates  for  multi-class 
classification  by  pairwise  coupling.  J  Mach  Learn  Res  2004; 5: 
975-1005. 

33.  Jayram  G,  Eggener  SE.  Patient  selection  for  focal  therapy  of 
localized  prostate  cancer.  Curr  Opin  Urol  2009;19:268-273. 

34.  Arora  R,  Koch  MO,  Eble  JN,  Ulbright  TM,  Li  L,  Cheng  L.  Hetero¬ 
geneity  of  Gleason  grade  in  multifocal  adenocarcinoma  of  the 
prostate.  Cancer  2004;100:2362-2366. 

35.  Ward  AD,  Crukley  C,  McKenzie  C,  et  al.  Registration  of  prostate 
magnetic  resonance  images  to  digital  histopathology  images.  Proc 
Prostate  Cancer  Imaging.  2010;66-76. 

36.  Mazaheri  Y,  Bokacheva  L,  Kroon  DJ,  et  al.  Semi-automatic 
deformable  registration  of  prostate  MR  images  to  pathological  sli¬ 
ces.  J  Magn  Reson  Imaging  2010;32:1149-1157. 

37.  Zhan  Y,  Ou  Y,  Feldman  M,  Tomaszeweski  J,  Davatzikos  C,  Shen 
D.  Registering  histologic  and  MR  images  of  prostate  for  image 
based  cancer  detection.  Acad  Radiol  2007;14:1367-1381. 

38.  Jacobs  MA,  Windham  JP,  Soltanian-Zadeh  H,  Peck  DJ,  Knight 
RA.  Registration  and  warping  of  magnetic  resonance  images  to 
histological  sections.  Med  Phys  1999;26:1568-1578. 


738 


IEEE  TRANSACTIONS  ON  MEDICAL  IMAGING,  VOL.  31,  NO.  3,  MARCH  2012 


Use  of  Needle  Track  Detection  to  Quantify 
the  Displacement  of  Stranded  Seeds 
Following  Prostate  Brachytherapy 

Julio  R.  Lobo,  Mehdi  Moradi,  Nick  Chng,  Ehsan  Dehghan,  William  J.  Morris,  Gabor  Fichtinger,  and 

Septimiu  E.  Salcudean* 


Abstract — We  aim  to  compute  the  movement  of  permanent 
stranded  implant  brachytherapy  radioactive  sources  (seeds)  in  the 
prostate  from  the  planned  seed  distribution  to  the  intraoperative 
fluoroscopic  distribution,  and  then  to  the  postimplant  computed 
tomography  (CT)  distribution.  We  present  a  novel  approach  to 
matching  the  seeds  in  these  distributions  to  the  plan  by  grouping 
the  seeds  into  needle  tracks.  First,  we  identify  the  implantation 
axis  using  a  sample  consensus  algorithm.  Then,  we  use  a  network 
flow  algorithm  to  group  seeds  into  their  needle  tracks.  Finally, 
we  match  the  needles  from  the  three  stages  using  both  their 
transverse  plane  location  and  the  number  of  seeds  per  needle.  We 
validated  our  approach  on  eight  clinical  prostate  brachytherapy 
cases,  having  a  total  of  871  brachytherapy  seeds  distributed  in  193 
needles.  For  the  intraoperative  and  postimplant  data,  99.31%  and 
99.41%  of  the  seeds  were  correctly  assigned,  respectively.  For  both 
the  preplan  to  fluoroscopic  and  fluoroscopic  to  CT  registrations, 
100%  of  the  needles  were  correctly  matched.  We  show  that  there  is 
an  average  intraoperative  seed  displacement  of  4.94  d=  2.42  mm 
and  a  further  2.97  d=  1.81  mm  of  postimplant  movement.  This 
information  reveals  several  directional  trends  and  can  be  used  for 
quality  control,  treatment  planning,  and  intraoperative  dosimetry 
that  fuses  ultrasound  and  fluoroscopy. 

Index  Terms — Computed-tomography  (CT),  minimum  cost  net¬ 
work  flow,  prostate  brachytherapy,  X-ray  fluoroscopy. 

I.  Introduction 

PROSTATE  brachytherapy  is  an  effective,  minimally  in¬ 
vasive  treatment  technique  for  men  with  prostate  cancer 
[1].  It  involves  transperineal  insertion  of  permanent  radioactive 
sources,  or  seeds,  into  the  prostate  using  needles.  A  satisfactory 

Manuscript  received  October  21,  2011;  accepted  November  21,  2011.  Date 
of  publication  December  06,  2011;  date  of  current  version  March  02,  2012.  The 
work  was  supported  by  Natural  Sciences  and  Engineering  Research  Council 
of  Canada  and  the  Canadian  Institutes  of  Health  Research.  The  work  of  G. 
Fichtinger  was  supported  as  a  Cancer  Care  Ontario  Research  Chair.  The  work 
of  M.  Moradi  was  supported  by  the  Congressionally  Directed  Medical  Research 
Program  under  W81XWH-10-1-0201.  The  work  of  E.  Dehghan  was  supported 
by  the  Ontario  Ministry  of  Research  and  Innovation  Postdoctoral  Fellowship. 
Asterisk  indicates  corresponding  author. 

J.  R.  Lobo  and  M.  Moradi  are  with  the  Department  of  Electrical  and  Computer 
Engineering,  University  of  British  Columbia,  Vancouver,  BC  V6T  1Z4,  Canada 
(e-mail:  juliol@ece.ubc.ca;  moradi@ece.ubc.ca). 

N.  Chng  and  W.  J.  Morris  are  with  the  Vancouver  Cancer  Center,  Vancouver, 
V5Z  4E6  BC  Canada  (e-mail:  nchng@bccancer.bc.ca;  jmorris@bccancer.bc. 
ca). 

E.  Dehghan  and  G.  Fichtinger  are  with  the  School  of  Computing,  Queen’s 
University,  Kingston,  ON  K7L  3N6,  Canada  (e-mail:  ehsand@cs.queensu.ca; 
gabor@cs.queensu.ca). 

*S.  E.  Salcudean  is  with  the  Department  of  Electrical  and  Computer  En¬ 
gineering,  University  of  British  Columbia,  Vancouver,  BC  V6T  1Z4,  Canada 
(e-mail:  tims@ece.ubc.ca). 

Color  versions  of  one  or  more  of  the  figures  in  this  paper  are  available  online 
at  http://ieeexplore.ieee.org. 

Digital  Object  Identifier  10.1109/TMI.2011.2178254 


implant  quality  requires  the  target  volume  to  be  sufficiently  irra¬ 
diated  while  sparing  noncancerous  tissue.  Poor  implant  quality 
can  arise  due  to  errors  in  the  seed  positions  caused  by  needle  de¬ 
flection  and  prostate  movement  during  insertion,  as  well  as  sub¬ 
sequent  seed  displacement  due  to  inflammation  of  the  prostate 
after  operation.  The  American  Brachytherapy  Society  identifies 
the  need  to  assess  the  impact  that  seed  displacement  has  on  the 
effectiveness  of  the  implant  and  recommends  routine  postim¬ 
plant  dosimetric  analysis  for  all  patients  [2]. 

Measurement  of  the  displacement  of  seeds  after  implantation 
has  been  performed  by  several  research  groups.  Many  of  the 
published  papers  on  measuring  seed  displacements  and  move¬ 
ment  use  manual  seed  detection  [3]— [7].  The  introduction  of  au¬ 
tomatic  methods  for  plan  reconstruction  [8],  [9]  is  likely  to  fa¬ 
cilitate  postimplant  dosimetry. 

Pinkawa  et  al.  investigated  postimplant  seed  migration  by 
looking  at  the  dose  levels  over  a  30  day  period  [10],  [11]. 
Manual  localization  and  contouring,  without  specific  seed 
identification  between  datasets,  was  required  in  this  study.  The 
individual  seed  displacements,  rather  than  the  overall  dose 
change,  was  quantified  in  Usmani  et  al .,  in  which  the  seed  po¬ 
sitions  relative  to  fiducial  markers  were  compared  in  repeated 
postimplant  computed-tomography  (CT)  images  [12].  In  Chng 
et  al .,  seeds  segmented  from  CT  data  were  automatically 
grouped  into  their  respective  needles  using  a  minimum  cost 
network  flow  algorithm  [13]  following  a  coarse  registration 
performed  by  iterating  through  all  trajectory  angles  of  every 
seed.  The  cost  function  consisted  of  complex  correspondence 
functions  used  to  compute  the  positional  error,  trajectory  angle 
error,  and  the  strand  spacing  error.  Once  the  needle  tracks  were 
detected,  a  nonlinear  optimization  algorithm  called  Graduated 
Assignment  was  then  used  to  match  the  entire  “graphs”  created 
by  the  needle  track  networks  [9].  Accurate  dose  changes  could 
be  computed  based  on  the  seed  positions. 

The  above  investigations  made  use  of  postoperative  data,  and 
could  only  find  the  seed  migration  due  to  the  combined  effect  of 
1)  seed  misplacement  at  the  time  of  the  implant,  and  2)  postim¬ 
plant  inflammation. 

We  have  combined  several  techniques,  including  the  use  of 
an  iterative  best  line  detection  algorithm  and  network  flow  algo¬ 
rithms,  to  formulate  a  novel  method  to  accurately  identify  corre¬ 
sponding  brachytherapy  seeds  in  different  datasets  to  perform  a 
seed  displacement  analysis.  Our  method  extends  the  grouping 
of  seeds  into  needles  approach  by  simplifying  the  cost  func¬ 
tion  components  so  that  it  is  effective  over  multiple  data  types. 
We  compare  seed  locations  obtained  from  intraoperative  fluoro- 
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based  images  both  to  preplan  data  and  to  postimplant  CT  data. 
The  two  comparisons  are  done  separately  which  allows  us  to 
investigate  and  distinguish  between  seed  misplacement  during 
surgery  and  postimplant  movement.  Our  postimplant  analysis 
uses  CT  images  taken  immediately  after  surgery  so  this  study 
does  not  look  at  long  term  seed  migration,  although  the  same 
techniques  described  here  can  be  used  for  CT  data  taken  at  later 
times  to  do  so.  The  comparison  with  intraoperative  seed  po¬ 
sitions,  however,  does  allow  an  investigation  of  the  effect  of 
change  in  patient  pose  which  changes  from  dorsal  lithotomy 
during  the  implant  procedure  to  horizontal  recumbent  during  CT 
imaging.  This  has  also  not  been  analyzed  using  individual  seed 
displacements  before.  Furthermore,  we  propose  that  given  find¬ 
ings  from  Moradi  et  al  who  show  that  given  a  complete  recon¬ 
struction  of  the  fluoroscopic  data  and  identification  of  the  seeds, 
registration  can  be  performed  with  an  incomplete  reconstruc¬ 
tion  of  the  seed  cloud  in  ultrasound  data  [13],  we  can  use  our 
methods  to  register  the  seed  displacements  within  the  prostate 
volume. 

We  have  developed  a  new  technique  for  registration,  which 
is  a  required  step  for  accurate  computation  of  seed  displace¬ 
ment.  We  stress  that  a  full  deformable  registration  by  matching 
individual  seeds  is  not  desired  as  this  can  remove  the  trends 
in  displacement  that  are  to  be  measured.  Therefore,  instead  of 
using  individual  seed  positions,  our  method  uses  two  global  av¬ 
erage  parameters  to  register  the  different  datasets.  These  param¬ 
eters  are  the  average  implantation  axis  and  the  seed  cluster  cen¬ 
troid  position.  Aligning  datasets  to  the  same  average  parame¬ 
ters  keeps  local  trend  information,  while  bringing  the  datasets 
into  the  same  coordinate  system  so  that  the  measured  values  are 
actual  distances.  For  the  implantation  axis  detection  we  use  an 
iterative  sampling  algorithm  to  obtain  a  ranked  list  of  potential 
needle  tracks  in  images.  The  average  of  the  highest  ranking  few 
needle  tracks  is  used  to  determine  the  implant  axis  direction. 
This  can  be  done  on  each  dataset  independently  removing  the 
need  for  an  exhaustive  iterative  closest  point  registration  [14] 
comparing  two  datasets  at  a  time. 

A  new  method  of  matching  detected  needles  between  datasets 
is  also  presented.  Another  network  flow  algorithm  is  used  where 
the  needles  are  matched  using  only  the  location  of  the  needle 
intersections  with  the  transverse  plane  of  implantation,  and  the 
number  of  seeds  in  each  given  needle.  The  same  needle  inter¬ 
sections  can  be  used  to  register  ultrasound  data  to  fluoroscopic 
data  leading  to  intraoperative  dosimetry. 

Our  registration  technique  ignores  any  offset  rotations  about 
the  insertion  axis  itself.  However,  we  have  developed  a  tech¬ 
nique  to  automatically  detect  these  offset  rotations  by  finding 
the  rotation  that  gives  the  minimum  average  seed  displacement. 
The  seed  displacements  must  be  recomputed  for  every  new  rota¬ 
tion  until  a  minimum  is  found,  but  the  insertion  axis  remains  the 
same  and  the  seeds  are  already  matched  between  datasets  and  so 
the  time  complexity  is  increased  only  by  a  constant  factor. 

Since  our  institution  only  uses  stranded  implants  for 
brachytherapy,  this  study  focuses  on  measuring  seed  displace¬ 
ments  during  and  after  stranded  seed  implantation.  Although 
it  is  possible  to  apply  our  methods  to  loose  seeds,  since  they 
are  usually  inserted  in  needle  tracks  with  regular  spacers, 
studies  have  shown  that  stranded  seeds  provide  better  dose 


coverage  [15]  and  less  seed  loss  [16]  without  compromising 
the  biochemical  no  evidence  of  disease  (bNED)  [17].  As  a 
consequence,  stranded  seeds  are  used  in  about  50%  of  cases  in 
the  United  States  and  in  over  80%  of  cases  around  the  world. 
Therefore  methods  tuned  to  work  on  stranded  implants  alone 
do  impact  a  large  proportion  of  the  brachytherapy  field. 

In  this  paper,  we  will  refer  to  the  difference  between  pre¬ 
plan  seed  locations  to  intraoperative  seed  location  as  “misplace¬ 
ment,”  and  the  difference  between  intraoperative  to  postimplant 
seed  locations  as  “movement.”  The  term  “displacement”  will  be 
used  to  describe  both  differences.  Although  the  term  “misplace¬ 
ment”  might  imply  that  this  method  can  distinguish  between 
tissue  deformation  and  physical  implant  misplacement  this  is 
not  the  case  since  we  do  not  have  any  information  about  the 
tissue.  The  misplacement  is  purely  the  difference  in  locations 
of  the  seeds  from  the  preplan  data  to  the  intraoperative  data. 

Preliminary  results  of  this  work  are  presented  in  Lobo  et  al 
[18].  This  paper  contains  a  significantly  expanded  analysis  and 
algorithmic  details.  Furthermore,  the  matching  and  registration 
details  are  updated  and  improved.  The  paper  first  describes 
the  imaging  used  and  the  subsequent  seed  localization.  The 
methods  required  to  match  seeds  in  different  datasets  and  to 
compute  the  seed  displacements  are  then  described.  Next, 
the  results  showing  average  displacements  and  trends  are 
presented.  Finally,  an  analysis  of  the  results  is  given  in  the 
conclusion. 

II.  Data  Collection  and  Discrete  Tomography 
Approach 

Datasets  from  eight  patients  undergoing  low-dose  prostate 
brachytherapy  were  collected  at  the  British  Columbia  Cancer 
Agency,  Vancouver,  Canada,  after  Institutional  Research  Ethics 
Board  approval  and  patient  consent  were  acquired.  For  each  of 
the  patients,  the  treatment  plan  was  generated  using  the  Variseed 
planning  software  (Varian  Medical,  Palo  Alto,  CA),  and  the 
planned  seed  positions,  referred  to  as  the  preoperative  plan  or 
preplan  were  exported  from  it.  The  preplan  data  is  obtained 
from  the  planned  brachytherapy  template  needle  position  and 
depth,  and  was  modified  to  account  for  changes  made  by  the  ra¬ 
diation  oncologists  during  the  procedure. 

The  postimplant  CT  was  acquired  approximately  1  h  after  the 
intervention.  The  coordinates  of  the  postimplant  CT,  referred  to 
as  the  postimplant ,  were  also  exported  from  the  VariSeed  plan¬ 
ning  software,  as  described  in  Chng  et  al  [9]. 

Intraoperative  seed  positions  were  reconstructed  from  fluo¬ 
roscopic  data  acquired  immediately  after  implanting  all  the  re¬ 
quired  seeds.  A  series  of  five  fluoroscopic  images  were  taken, 
rotating  the  C-arm  from  approximately  —10°  to  10°,  in  the 
transverse  plane.  The  joint  angle  was  recorded  for  each  image 
using  a  digital  protractor  secured  to  the  C-arm.  Fig.  1  illus¬ 
trates  the  C-arm  set  up  from  the  operating  room.  An  example 
CT  image  showing  real  brachytherapy  seeds  is  also  shown  in 
the  inset  for  comparison. 

A  seed-projection  matching  approach  with  motion  compen¬ 
sation  was  used  to  obtain  seed  reconstructions  from  the  intraop¬ 
erative  data  [19].  The  method  uses  a  network  flow  solution  to 
minimize  the  distance  between  back-projected  lines  from  seed 
projections  [20],  and  motion  compensation  that  recovers  the 
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Fig.  1.  Prostate  Brachytherapy  setup  with  C-arm  used  to  take  intraoperative  Fig.  2.  Full  seed  displacement  algorithm, 
fluoroscopic  data.  The  coordinate  system  that  is  used  is  also  shown.  Inset:  An 
example  2D  CT  cross  section  image  for  comparison. 


C-arm  pose  without  the  need  for  fiducial  markers  or  auxiliary 
trackers.  Its  results  were  carefully  validated  in  simulations,  im¬ 
plant  phantoms  and  patient  data  [19]. 

III.  Methods 

Seeds  are  matched  between  datasets  by  first  finding  the  needle 
tracks  they  belong  to.  The  minimum  cost  algorithm  described 
by  Chng  et  al  [9]  is  used  to  find  these  needles  tracks  in  the  dif¬ 
ferent  datasets.  The  intersections  of  the  needles  with  the  trans¬ 
verse  insertion  plane  are  computed  next  and  a  different  min¬ 
imum  cost  network  flow  algorithm  is  used  to  match  these  in¬ 
tersections  between  datasets.  With  the  intersections  and  there¬ 
fore  needles  matched,  the  seeds  in  each  needle  are  then  labeled 
and  seed  displacements  are  computed.  The  approach  is  summa¬ 
rized  in  Fig.  2.  For  all  the  descriptions  and  results  presented  in 
this  paper  we  employ  a  conventional  right-handed  coordinate 
system  in  which  the  z  axis  is  also  the  insertion  axis  and  runs 
inferior  to  superior  and  the  x  axis  runs  posterior  to  anterior,  as 
shown  in  Fig.  1.  The  terms  yaw,  pitch  and  roll  refer  to  rota¬ 
tions  about  the  x  axis,  y  axis  and  z  axis,  respectively.  Detailed 
descriptions  of  the  methods  are  presented  in  the  following  five 
subsections. 

A.  Registration  Using  Insertion  Axis  Detection 

Previous  work  has  shown  that  to  accurately  compute  the  dis¬ 
placements,  it  is  necessary  to  put  the  different  datasets  into  cor¬ 
respondence  [9].  Full  point  to  point  registration  that  can  remove 
local  trends  is  not  desired.  Instead,  we  carry  out  the  registration 
using  average  components  of  the  different  clusters.  Each  dataset 
is  globally  translated  and  rotated  so  that  the  seed  cluster  centroid 
position  is  located  at  the  origin  and  the  average  implantation 
axis  is  aligned  with  the  £  axis. 


Usmani  et  al  show  that  the  seed  cluster  centroid  position 
can  be  used  to  register  the  datasets  since  it  remains  at  the  same 
position  even  with  considerable  postimplant  migration  [12]. 
Matching  the  implantation  axes  in  the  datasets  removes  global 
pitch  and  yaw  differences.  Local  pitch  and  yaw  trends  are 
expected  and  can  be  measured  since  only  the  average  insertion 
direction  is  used.  No  roll  trends  are  expected  so  postprocessing 
is  used  to  remove  any  slight  roll  differences.  The  axis  detection 
is  carried  out  on  each  dataset  separately  which  reduces  the 
computation  that  would  be  required  to  compute  costs  from 
looking  at  two  datasets  simultaneously,  as  would  be  carried  out 
with  an  iterative  closest  point  method. 

The  main  component  of  our  registration  process  involves 
finding  and  aligning  the  average  implantation  axes  in  different 
datasets.  Our  iterative  best  line  algorithm  to  find  the  axis 
of  insertion  is  similar  to  the  RANSAC  (RANdom  SAmple 
Consensus,  [21])  algorithm  which  involves  fitting  lines  to 
pairs  of  randomly  chosen  points  and  scoring  each  line.  The 
highest  scored  lines  are  then  kept  as  needles.  However,  our 
implementation  has  no  random  component.  This  is  due  to  the 
fact  that  an  exhaustive  search  on  all  possible  pairs  of  points  for 
this  application  is  not  computationally  demanding.  This  gives 
an  effective  way  of  finding  a  majority  of  the  needles  but  does 
not  consistently  detect  all  of  them.  However,  as  we  will  show, 
this  allows  us  to  find  the  implantation  axis,  which  coincides 
with  the  ultrasound  transducer  axis. 

Pairs  of  seeds  are  used  to  fit  straight  lines  that  define  po¬ 
tential  needle  directions.  Implausible  pairs,  calculated  based  on 
whether  the  lines  are  within  a  certain  angle  threshold  from  the  z 
axis,  are  rejected  immediately  saving  unnecessary  calculations. 
Seeds  belonging  to  the  same  needle  may  not  necessarily  lie  on 
a  straight  line,  as  needle  tracks  can  exhibit  significant  curva¬ 
ture  which  is  not  easily  parameterizable,  as  illustrated  in  Fig.  3. 
Therefore,  a  tolerance  region  is  required  when  matching  seeds 
to  a  single  line  modeling  the  needle. 
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Fig.  3.  Illustration  of  several  possible  curvatures  that  can  be  detected  for  a 
needle  by  using  a  Gaussian  cone  rather  than  a  tube  with  fixed  radius. 


We  chose  a  Gaussian  cone  as  a  tolerance  region  for  each  po¬ 
tential  needle  direction  line  since  it  applies  tight  constraints  to 
potential  tracks  when  they  are  close  to  the  insertion  plane  and 
relaxes  them  farther  away  where  there  is  more  deviation.  The 
depth  of  focus  can  be  varied  to  give  a  sharper  cut-off  for  higher 
angles  than  a  simple  cone  tolerance  region  gives.  A  given  seed, 
i,  only  contributes  to  the  score  if  it  is  closer  than  a  distance 
to  the  potential  line.  Wi  is  the  “waist”  of  the  tolerance  which 
increases  with  the  distance  from  the  insertion  plane,  a;  and  is 
computed  as 


Wi  =  W  0 


(1) 


where  wo  is  the  minimum  waist  size  and  r  is  the  distance  at 
which  Wi  =  wo  \/2 .  They  control  the  waist  at  the  insertion  plane 
and  the  rate  of  expansion  of  the  Gaussian  tolerance  cone.  Fi¬ 
nally,  since  the  spacing  in  stranded  seeds  can  only  be  a  multiple 
of  1 0  mm  we  remove  any  seeds  that  do  not  satisfy  this  require¬ 
ment  with  a  tolerance  of  ±2  mm.  The  spacing  between  seed  i 
and  an  adjacent  seed  j  is  denoted  spij .  This  eliminates  the  cases 
when  seeds  from  other  needle  tracks  fall  within  the  Gaussian 
cone  when  the  waist  size  is  large. 

Letting  di  be  the  distance  of  seed  i  from  the  potential  needle 
axis,  the  score  becomes 

Score  =  e~di/Wi .  (2) 

{i:di  <Wi  ,\spij  — 10 1  <2Vj?} 


As  shown  in  Section  IV-A,  the  iterative  best  line  detection  al¬ 
gorithm  finds  a  majority  of  the  needles.  However,  in  this  work, 
we  used  this  algorithm  primarily  to  find  the  axis  of  insertion  of 
needles  which  is  an  important  piece  of  information  to  ensure  the 
accuracy  of  the  network  flow  algorithm.  By  relaxing  the  con¬ 
straints  and  therefore  allowing  needles  with  a  larger  deviation 
from  the  z  axis,  we  were  able  to  find  the  implantation  axis  by 
averaging  the  direction  vectors  of  the  highest  scoring  eight  nee¬ 
dles.  Eight  needles  were  chosen  instead  of  using  a  cutoff  calcu¬ 
lated  for  each  patient  for  consistency.  Each  of  the  seed  clusters 
were  then  rotated  so  that  the  implantation  axis  was  parallel  to 
the  z  axis.  The  preplan  data  has  the  transducer  axis  already  par¬ 
allel  to  the  2:  axis,  and  obviously  did  not  require  any  rotation. 

B.  Assigning  Seeds  to  Needle  Tracks 

Once  the  dataset  is  rotated  so  that  it  is  aligned  with  the  im¬ 
plantation  axis,  it  is  possible  to  use  a  needle  detection  algorithm 
with  tighter  restrictions.  The  curvature  problem,  however,  still 


remains.  Therefore,  instead  of  looking  at  parameterizable  lines 
the  seeds  are  viewed  as  nodes  within  a  network  that  allow  a 
single  unit  of  flow  through  them.  Each  arc  between  two  nodes 
carries  a  cost  for  a  unit  flow.  The  Matlab  Toolbox  “matlog”  de¬ 
veloped  by  M.  G.  Kay  at  the  North  Carolina  State  University 
was  used  to  determine  the  flow  lines,  which  were  tuned  to  be 
the  needle  tracks,  that  resulted  in  the  lowest  total  cost1  [22]. 

To  summarize  the  process:  first,  two  new  nodes  are  created, 
one  start  node  to  supply  all  the  flow  and  one  end  node  to  sink 
it.  All  the  seeds  are  defined  as  transshipment  nodes,  located  be¬ 
tween  the  start  and  end  node,  where  the  inflow  must  equal  the 
outflow.  Each  node  is  initially  given  the  same  negative  cost  so 
that  they  are  all  equally  likely  to  be  used,  while  a  cost  func¬ 
tion  is  used  to  define  the  costs  of  flow  going  through  every  arc 
joining  two  nodes,  from  the  source  node  to  the  sink  node.  This 
cost  function  allows  for  the  correct  needle  track  detection.  Since 
each  arc  is  independent  of  the  previous  one,  there  are  no  restric¬ 
tions  as  to  the  direction  of  the  flow  based  on  the  shape  of  the 
curve  before  the  node  which  allows  for  the  changes  in  direction 
that  sometimes  occur.  A  cost  function  with  two  components  is 
used.  These  are  the  angle  that  a  given  flow  line  makes  with  the 
average  insertion  axis,  and  the  length  of  the  unit  of  flow — which 
should  be  a  multiple  of  the  1 0  mm.  Although  each  component 
function  has  a  complex  form  derived  in  Chng  et  al.  [9]  we  use 
much  simpler  functions  that  still  give  accurate  results. 

The  angle  cost  component  is  defined  as  follows: 


ACij  =  (e|9”l/e°  -  l)4 

and  the  spacing  component  of  the  cost  is 

E!  6(sii  -  ns°) 


a  e 


~{sij/sv)2 


(3) 


(4) 


where  Sij  is  the  spacing  between  node  i  and  node  j  and  \6ij\  is 
the  angle  the  line  between  the  two  nodes  makes  with  the  z  axis. 
The  symbol  a  denotes  convolution.  The  spacing  cost  is  defined 
as  a  Gaussian  with  a  variance  of  sv  convolved  with  a  train  of 
delta  functions  to  give  negative  peaks  at  integer  multiples  of 
the  smallest  spacing  (s0  =  10  mm).  The  decaying  exponential 
with  a  variance  of  Sd  =  10  mm  is  aimed  at  penalizing  larger 
seed  spacing.  It  decreases  the  magnitude  of  the  spacing  cost  by 
a  factor  1/e  for  every  successive  integer  multiple  of  the  spacing 
So  •  Note  that  the  contribution  of  each  successive  Gaussian  peak 
to  the  cost  decreases  exponentially  as  n  increases.  However,  in 
practice,  it  is  extremely  unlikely  that  the  seed  spacing  in  a  given 
needle  is  40  mm  or  more  so  we  only  sum  until  n  —  3.  The 
fourth  power  for  the  angle  cost  ensures  that  the  cost  increases 
rapidly  for  increasing  angles.  Oo  and  sv  are  tunable  parame¬ 
ters  that  define  how  “strict”  the  function  is  with  respect  to  the 
angle  and  seed  spacing,  respectively.  For  the  fluoroscopic  data, 
0o  was  set  to  43.0°  and  sv  to  1.2  mm.  A  large  angle  variance 
was  needed  since  several  needle  tracks  were  visibly  curved  and 
rotated  but  tighter  restrictions  on  the  seed  separation  ensured 
that  true  tracks  were  identified.  In  contrast,  for  the  postimplant 
CT  data,  Oo  was  set  to  31.5°  and  sv  to  3  mm.  Here,  the  seeds 
migrated  a  little  so  the  seed  separation  requirement  was  relaxed. 


1http://www.ise.ncsu.edu/kay/matlog/ 
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Angle  cost  Seed  spacing  cost 


Fig.  4.  Graphs  of  the  angle  (left)  and  spacing  (right)  cost  functions  showing 
high  costs  for  higher  angles  and  low  costs  for  multiples  of  the  smallest  seed 
spacing  (10  mm).  The  tunable  parameters  for  the  angle  variance,  0Q,  and  the 
spacing  tolerance,  s  v ,  are  shown  with  a  box  around  them.  The  spacing  constants, 
s0  and  sd  are  also  shown. 

Fig.  4  graphically  shows  the  angle  and  seed  spacing  cost  func¬ 
tions  with  the  different  parameters  and  constants. 

The  total  cost  is  obtained  by  summing  the  two  costs  with  a 
weighting  parameter,  7,  on  the  angle  cost  term.  This  parameter 
controls  which  component  affects  the  cost  most.  We  found  that 
increasing  the  sensitivity  of  the  function  to  the  spacing  of  the 
seeds  by  decreasing  the  value  of  7  produced  the  best  results. 
A  lower  sensitivity  to  the  angle  was  needed  to  allow  for  the 
observed  variation  in  the  tracks.  Therefore,  for  the  intraopera¬ 
tive  fluoroscopic  and  postimplant  CT  data,  7  was  set  to  0.3.  For 
the  preplan  data,  however,  7  is  set  to  1 .2  so  that  the  function  is 
weighted  more  to  the  angle  cost  so  that  any  angle  deviation  from 
the  z  axis  has  a  much  higher  cost.  The  final  objective  function  is 

Cij=1ACij  +  SCij.  (5) 

C.  Needle  Matching 

The  needle  detection  technique  summarized  above  leverages 
intra-strand  seed  spacing,  enforced  physically  by  the  stranded 
seed  product  routinely  used  in  our  implants,  as  the  classification 
characteristic  that  is  least  sensitive  to  noise.  This  simplifies  a 
matching  of  ~  100  seeds  to  one  of  ~  20  needles.  This  section 
explains  how  the  needle  matching  across  datasets  is  carried  out. 

After  rotating  the  seed  clusters  and  grouping  the  seeds  into 
needles,  the  insertion  plane  intersection  of  each  needle  is 
computed.  The  insertion  plane  is  the  transverse  plane  passing 
through  the  most  inferior  seed  (at  the  apex  of  the  prostate). 
Matching  is  then  performed  using  another  minimum  cost  net¬ 
work  flow  algorithm.  Each  node  of  the  network  contains  the  x , 
y  coordinates  of  an  intersection  as  well  as  the  number  of  seeds 
that  were  in  the  needle.  In  this  case,  instead  of  adding  a  source 
and  sink  node,  all  the  nodes  from  one  type  of  intersection  data 
are  defined  as  source  nodes  and  all  the  nodes  from  the  other 
type  are  defined  as  the  sink  nodes  (there  are  no  transshipment 
nodes).  The  cost  of  a  unit  flow  from  an  intersection  from  one 
data  type  to  that  of  another  one  is  defined  as 

MatchCost^-  =  dij  +  (TV;  —  Nj  +  l)4.  (6) 

Here  MatchCost;j  is  the  cost  of  matching  a  needle  intersec¬ 
tion,  z,  from  one  dataset  to  another  needle  intersection,  j,  in  an¬ 
other  dataset.  d%3  is  the  distance  between  intersection  i  and  in¬ 
tersection  j,  while  Ni  and  Nj  are  the  number  of  seeds  in  needle 


Intra-operative  and  post-implant  needle  insertion  plane  intersections 


Fig.  5.  Insertion  plane  intersections  for  one  patient  showing  why  the  number 
of  seeds  per  needle  is  also  needed  in  the  cost  function. 

i  and  needle  j ,  respectively.  We  add  1  in  the  term  Nt  —  Nj  +  1 
to  emphasize  the  difference  in  seed  numbers,  even  when  that 
difference  is  only  1 .  A  low  cost  is  assigned  if  the  corresponding 
needle  intersection  is  close  by  and  also  has  the  same  number 
of  seeds  in  the  needles.  d0  is  the  parameter  used  to  control  the 
effect  of  the  spacing  on  the  cost.  Since  the  number  of  needles 
in  the  two  datasets  can  now  be  set  a  priori  the  algorithm  tries 
different  pairings  until  all  are  matched  at  the  lowest  cost.  Fig.  5 
shows  the  an  example  of  insertion  plane  intersections.  The  red 
lines  show  some  of  the  correct  matches.  Note  that  simply  using 
a  closest-to  match  would  not  work. 

D.  Seed  Displacement  Computation 

Each  seed  position  is  compared  to  its  corresponding  seed 
from  a  different  data  type  to  obtain  a  displacement  vector.  An 
entire  dataset  of  each  seed  displacement  vector  in  each  patient 
was  computed  to  find  trends.  Sets  of  displacement  vectors  were 
generated  both  for  intraoperative  misplacement  and  for  postim¬ 
plant  movement. 

Scalar  distances  were  used  to  quantify  the  motion.  The  total 
average  distance  was  computed  for  each  patient  as  well  as  for 
all  the  patients.  The  average  distance  was  also  computed  for  dif¬ 
ferent  regions  of  the  prostate  by  dividing  the  entire  volume  into 
27  subregions,  where  each  axis  was  divided  into  three  sections. 

We  also  looked  at  the  direction  of  seed  motion.  This  was  done 
by  computing  an  average  displacement  vector  in  each  of  the 
subregions.  These  displacement  vectors  were  used  to  analyze 
the  trends  the  direction  of  motion  in  each  region. 

E.  Postprocessing  to  Fine  Tune  Dataset  Registration 

Section  III-A  describes  the  registration  used  to  correct  dif¬ 
ferences  in  the  global  pitch  and  yaw  which  allows  local  trends 
to  be  measured.  These  trends  are  expected  to  be  as  a  result  of 
several  mechanisms  including  needle  deflection,  prostate  rota¬ 
tion,  oncologist  preferences,  inflammation  and  change  in  pa¬ 
tient  pose.  None  of  these,  however,  are  expected  to  create  any 
local  roll  trends.  The  global  differences  in  roll,  due  to  slight 
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Roll  angle  (deg) 


Roll  angle  (deg) 


Preplan  to  intra-operative  comparisons 


Intra-operative  to  post-implant  comparisons 


Fig.  6.  Graphs  showing  how  the  average  displacement  distances  varied  with 
different  roll  values  for  the  intraoperative  data.  The  preplan  to  intraoperative 
comparisons  are  shown  on  the  left  and  intraoperative  to  postimplant  compar¬ 
isons  are  shown  on  the  right. 


TABLE  I 

Needle  Track  Detection  Using  the  Iterative 
Best  Line  Detection  Algorithm 


Patient 

Number  of 
incorrectly 
assigned 
seeds 

Number  of 
seeds 
inserted 

Percent  seeds 
correctly 
assigned 

1 

21 

105 

80% 

2 

14 

105 

86.67% 

3 

16 

102 

84.31% 

4 

25 

122 

79.51% 

5 

8 

104 

92.31% 

6 

13 

115 

88.70% 

7 

21 

100 

79% 

8 

8 

118 

93.22% 

differences  in  imaging  angles,  were  therefore  automatically  re¬ 
moved  after  the  displacement  calculations.  This  was  done  by 
computing  the  average  displacement  distance  for  each  patient 
for  different  amounts  of  roll  of  the  intraoperative  data,  which 
is  used  in  both  comparisons.  Fig.  6  shows  how  the  average  dis¬ 
placement  distance  varied  with  roll  angle  for  each  patient.  The 
roll  angle  which  gives  the  minimum  average  displacement  dis¬ 
tance  is  taken  as  the  value  which  aligns  the  roll  between  the 
datasets  and  the  displacements  with  this  roll  are  reported  in  this 
paper. 


IV.  Results  and  Discussions 

This  section  will  first  present  the  verification  of  the  individual 
processes  described  in  the  methods  followed  by  the  actual  seed 
displacement  results.  These  results  include  quantitative  values 
for  the  displacement  for  each  patient  and  also  for  each  region 
of  the  target  volume  (the  prostate).  A  qualitative  description  of 
the  direction  of  displacement  is  given  following  a  numerical 
analysis  to  assess  the  presence  of  trends. 

A.  Finding  the  Transducer  Axis 

The  iterative  best  line  detection  algorithm  resulted  in  needle 
detection  outcomes  reported  in  Table  I.  The  needle  detection 
results  vary  from  79.0%  to  93.22%  of  seeds  that  were  correctly 
assigned.  Based  on  the  average  vector  directions  of  the  eight 
highest  scoring  needles,  we  found  the  implantation  axis  and  ro¬ 
tated  the  cluster  of  seeds  so  that  it  was  aligned  with  the  z  axis. 
Fig.  7  shows  that  the  alignment  of  the  implantation  axis  with  the 
2; -axis  was  achieved  for  both  intraoperative  fluoroscopic  data 
and  postimplant  CT  data. 


Unrotated  Cluster  of  intra-operative  data  Unrotated  Cluster  of  post-implant  data 
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Fig.  7.  Finding  the  implantation  axis  using  an  iterative  best  line  detection  for 
intraoperative  fluoroscopic  data. 


The  transducer  axis  is  assumed  to  have  the  same  pitch  as  the 
most  posterior  needles.  Although  Fig.  7  only  shows  the  sagittal 
view,  it  is  worth  noting  that  the  algorithm  performed  3D  rota¬ 
tions  and  not  just  a  single  rotation  about  the  y  axis. 

B.  Needle  Track  Detection 

The  full  algorithm  was  tested  on  eight  patient  datasets.  The 
number  of  needles  and  number  of  correctly  assigned  seeds  were 
determined  by  comparing  the  result  to  the  preplan  (potentially 
modified  by  the  oncologist  during  operation).  Fig.  8  shows  the 
results  from  a  single  patient  for  all  three  data  types. 

Intersections  of  detected  needle  lines  at  the  insertion  plane 
were  used  to  compare  to  the  plan,  looking  at  the  number  of 
seeds  per  needle  as  well  as  the  expected  relative  positions.  This 
simplified  the  finding  of  potentially  incorrectly  assigned  seeds. 
Fig.  9  shows  an  example  of  a  plan  comparison.  The  black  el¬ 
lipses  show  how  the  comparison  can  be  used  to  identify  which 
seeds  were  not  assigned  correctly. 

The  needle  matching  for  the  preplan  data,  not  surprisingly, 
correctly  assigned  all  the  seeds  in  all  patients.  The  results  for 
the  intraoperative  and  postimplant  seed  to  needle  assignment 
are  summarized  in  Table  II  which  shows  the  percentage  of  seeds 
that  were  correctly  assigned. 

It  is  worthwhile  to  note  that  patients  3  and  5  had  two  spe¬ 
cial  load  needles  (needles  with  irregular  seed  spacing)  each.  As 
expected,  this  made  it  more  difficult  to  find  those  needles  and 
accounted  for  the  incorrect  seed  assignments  for  both  these  pa¬ 
tients.  Summarizing  the  results,  an  average  of  99.31%  of  the 
seeds  were  correctly  assigned  for  the  intraoperative  data,  while 
an  average  of  99.41%  of  the  seeds  were  correctly  assigned  for 
the  postimplant  data.  Once  again,  no  preplan  information  is  used 
in  the  needle  detection  and  the  algorithm  takes  between  1  to  2.5 
s  to  rotate  the  cluster  and  find  the  needles. 

C.  Inter-Dataset  Needle  Matching 

Before  continuing,  the  few  incorrectly  assigned  seeds  in  the 
various  datasets  were  manually  adjusted  to  be  grouped  with  the 
correct  needle.  Insertion  plane  intersections,  previously  used  to 
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Needle  detection  results  from  pre-plan  data 


Fig.  8.  Needle  detection  results  for  patient  4  for  preplan,  intraoperative  and 
postimplant  data. 


TABLE  II 

Summary  of  Needle  Track  Results  for  the  Both  Intraoperative 
and  Postimplant  Data,  on  Eight  Patients 


Patient 

Number  of 
seeds 
inserted 

Percent 
intra-op  seeds 
correctly 
assigned 

Percent 

post-imp  seeds 
correctly 
assigned 

1 

105 

100% 

100% 

2 

105 

100% 

98.10% 

3* 

102 

98.04% 

98.04% 

4 

122 

98.36% 

98.18% 

5* 

104 

98.08% 

100% 

6 

115 

100% 

100% 

7 

100 

100% 

100% 

8 

118 

100% 

100% 

*  patients  with  special  load  needles 
(needles  with  irregular  seed  spacing) 


Matched  intersection  points 


Fig.  10.  Intraoperative  to  postimplant  needle  matching  results  for  one  patient. 
Matching  needles  are  connected  with  the  red  lines. 


Fig.  9.  Comparison  of  fluoroscopic  intersection  data  with  preplan  data  showing 
some  incorrectly  assigned  seeds.  The  same  images  were  used  for  CT  to  preplan 
comparison. 


verify  the  needle  detection,  were  then  used  to  perform  needle 
matching  between  the  datasets.  The  matching  of  the  preplan  to 


the  intraoperative  data  and  the  intraoperative  data  to  the  postim¬ 
plant  data  still  required  verification  since  the  expected  displace¬ 
ment  made  the  insertion  plane  intersections  vary  considerably 
between  datasets.  The  method  described  earlier  achieved  a  cor¬ 
rect  needle  matching  for  all  the  datasets,  for  both  comparisons. 
Fig.  10  shows  how  the  intersection  data  displayed  in  Fig.  5 
was  correctly  matched.  The  red  lines  connect  needles  that  are 
matched  by  the  algorithm. 

D.  Calculation  of  Seed  Displacement 

With  the  needles  correctly  identified  and  matched  in  corre¬ 
sponding  data  sets  for  a  given  patient,  the  seeds  themselves 
could  be  directly  compared  which  gave  us  a  measure  of  the  seed 
displacements.  Figs.  1 1  and  12  show  the  movements  of  the  seeds 
between  the  preplan  and  intraoperative  data  and  between  the  in¬ 
traoperative  and  postimplant  data  for  one  of  the  patients,  respec¬ 
tively. 

1)  Average  Displacement  Magnitude:  The  3-D  Euclidean 
distance  that  every  seed  moved  between  datasets  was  computed. 
For  each  single  patient  the  average  distance  was  computed  as  the 
sum  of  all  the  distances  moved  divided  by  the  total  number  of 
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I ntra -operative  misplacement 


Fig.  11.  Preplan  to  intraoperative  seed  misplacement  results  for  patient  4. 


Post-implant  movement 


Fig.  12.  Intraoperative  to  postimplant  seed  movement  results  for  patient  4. 


TABLE  III 

Seed  Displacement  Results  for  Both  the  Preplan  to  Intraoperative 
Data  as  Well  as  the  Intraoperative  to  Postimplant  Data 


Patient 

Av.  misplacement 
distance 
(pre  to 
intra-op) 

/mm 

Av.  movement 
distance 
(intra-op 
to  post-imp) 
/mm 

1 

4.56±2.10 

3.76±1.89 

2 

6.17±2.44 

3.93±2.45 

3 

4.39±1.78 

3.16±1.78 

4 

4.08±2.25 

2.39±1.41 

5 

5.59±2.25 

2.85T1.73 

6 

4.89±2.18 

2.38±1.09 

7 

4.82±2.46 

1.72=1=0.75 

8 

5.12±3.02 

3.57=1=1.72 

Average 

4.94±2.42 

2.97=1=1.81 

seeds  inserted  into  the  patient.  The  total  average  over  all  eight 
patients  was  also  computed.  Table  III  summarizes  the  results. 

From  Table  III  the  average  displacement  was  significantly 
larger  for  preplan  to  intraoperative  case  than  intraoperative  to 
postimplant  data  (p  <  0.01,  n  =  871). 

The  error  in  our  localization  of  the  seeds  in  intraoperative 
fluoroscopy  data,  is  reported  to  be  less  than  0.9  mm  [19].  The 


TABLE  IV 

INTRAOPERATIVE  SEED  MISPLACEMENT  RESULTS  FOR  DIFFERENT 

Regions  Within  the  Prostate  Volume 


Average  misplacement  distance  (mm) 

left 

mid  y 

right 

inferior 

anterior 
mid  x 
posterior 

5.42  ±  1.41 
4.11  =L  2.19 
4.25  d=  1.52 

6.52  ±  2.63 
5.79  ±  2.41 
5.23  ±  1.76 

4.81  ±  2.41 
4.01  ±  2.44 
4.16  ±  2.05 

mid  2 

anterior 
mid  x 
posterior 

5.13  ±  1.51 
4.65  ±  2.14 
3.81  ±  1.68 

6.58  ±  3.07 
6.15  ±  3.17 
5.07  ±  1.97 

5.38  ±  2.92 
4.89  ±  2.36 
4.14  ±  2.88 

superior 

anterior 
mid  x 
posterior 

5.84  ±  1.25 
5.06  ±  2.20 
4.03  ±  1.93 

6.70  ±  2.76 
5.55  ±  2.58 
5.24  ±  2.36 

5.71  ±  2.09 
5.14  ±  2.51 
4.96  ±  3.01 

TABLE  V 

Postoperative  Seed  Movement  Results  for  Different 
Regions  Within  the  Prostate  Volume 


Average  distance  moved  (mm) 

left 

mid  y 

right 

inferior 

anterior 
mid  x 
posterior 

2.96  ±1.71 
2.99  ±  1.76 
2.66  ±  1.55 

3.67  ±  2.34 
2.82  ±  1.57 
2.58  ±  1.60 

3.16  ±  1.81 
3.34  ±  1.85 
3.26  ±  1.76 

mid  z 

anterior 
mid  x 
posterior 

2.68  ±  2.30 
3.03  ±  2.01 
3.15  ±  1.42 

3.29  ±  2.45 
1.60  ±  0.86 

2.30  ±  1.30 

3.13  ±  2.14 
2.59  ±  1.90 
2.98  ±  1.53 

superior 

anterior 
mid  x 
posterior 

3.28  ±  2.36 
3.04  ±  1.82 
3.10  ±  1.35 

2.89  ±  1.67 
2.16  ±  0.98 
2.34  ±  1.18 

3.85  ±  2.31 
2.56  ±  1.70 
3.04  ±  1.02 

calculated  displacement  is  therefore  not  due  to  errors  in  seed 
localization.  The  result  suggests  seed  displacement  due  to  on¬ 
cologist  preferences,  needle  deflection  and  prostate  movement 
during  needle  insertion,  seen  from  preplan  to  intraoperative  mis¬ 
placement  as  an  average  of  4.94  mm,  is  higher  than  displace¬ 
ment  caused  by  a  change  in  patient  pose  and  immediate  inflam¬ 
mation  (measured  as  intraoperative  to  postimplant  movement, 
an  average  of  2.97  mm). 

2)  Regional  Displacement  Magnitudes:  The  average  dis¬ 
placement  was  computed  for  each  of  the  27  subregions,  defined 
by  dividing  each  axis  into  three  sections,  to  quantify  the  motion 
in  each  region.  Tables  IV  and  V  show  these  distances.  In  the 
table,  the  different  transverse  slices  presented  from  inferior  to 
superior.  Each  transverse  slice  has  nine  distance  values. 

For  the  preplan  to  intraoperative  displacement,  the  seeds  near 
the  medial  line  of  the  prostate  (the  2  axis)  moved  slightly  more 
on  average.  Note  that  no  directional  information  can  be  drawn 
from  this.  For  intraoperative  to  postimplant  displacements,  there 
were  no  significant  differences  in  the  amount  of  motion  be¬ 
tween  the  different  subregions.  In  agreement  with  the  average 
patient  data,  the  intraoperative  misplacement  was  greater  than 
the  postimplant  movement  in  all  regions. 

3)  Displacement  Vectors:  It  was  noted  that  different  mag¬ 
nitudes  of  displacement  occurred  in  different  regions  of  the 
prostate.  We  computed  the  average  displacement  vectors  for 
each  of  the  27  subregions.  Tables  VI  and  VII  summarize  the 
general  displacement  directions  seen.  The  displacement  vec¬ 
tors  are  visually  illustrated  in  Figs.  13  and  14.  The  standard 
deviation  ellipsoids  are  drawn  in  separate  plots  where  each 
ellipsoid  is  centered  in  its  respective  region.  Note  that  the  scale 
is  doubled  for  the  postimplant  vectors  and  standard  deviations 
so  that  they  can  be  seen. 
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TABLE  VI 

Preplan  to  Intraoperative  Misplacement  Direction  Results. 
Values  With  a  Mean  Value  to  Standard  Deviation  Ratio 
Greater  Than  0.95  are  in  Boldface 


Average  misplacement  direction  vectors  ( x,y,z )  (mm) 

left 

mid  y 

right 

ant 

(0.33,0.54,-3.87) 

±(2.36,2.85,1.79) 

(-0.01,-1.47,-4.14) 

±(2.59,2.55,4.20) 

(-0.51,-1.59,-2.88) 

±(2.05,2.06,3.15) 

inf 

mid  x 

(-0.24,0.02,0.31) 

±(1.55,2.01,3.92) 

(0.94,0.31,2.03) 

±(1.48,2.16,5.56) 

(-0.70,-0.73,0.31) 

±(1.96,2.14,3.56) 

post 

(0.41,1.01,1.87) 

±(2.43,1.85,2.67) 

(0.21,-0.05,3.47) 

±(3.33,1.14,2.55) 

(1.40,0.07,0.55) 

±(2.61,1.78,3.22) 

ant 

(1.63,0.15,-3.83) 

±(1.53,2.47,1.85) 

(1.90,-0.69,-3.71) 

±(2.31,2.73,4.82) 

(0.42,-0.04,-4.06) 

±(2.48,1.82,3.46) 

mid  ^ 

mid  x 

(0.10,-0.97,0.70) 

±(2.09,1.87,4.15) 

(-0.13,-0.28,3.21) 

±(1.86,2.36,5.75) 

(-0.79,0.96,0.57) 

±(2.52,2.49,3.92) 

post 

(0.29,-0.07,1.45) 

±(2.12,2.12,2.57) 

(-0.60,0.54,3.46) 

±(2.83,1.41,2.74) 

(0.00,1.15,0.96) 

±(1.84,2.08,4.01) 

ant 

(3.07,-1.48,-3.51) 

±(2.02,2.47,1.51) 

(2.58,-0.60,-3.56) 

±(2.31,3.16,4.28) 

(1.68,2.34,-2.74) 

±(2.64,2.63,2.86) 

sup 

mid  x 

(0.12,-1.79,0.36) 

±(2.60,2.12,4.02) 

(0.11,0.62,2.58) 

±(2.36,2.35,4.72) 

(-0.70,2.07,0.51) 

±(2.77,2.80,3.53) 

post 

(-0.41,-0.40,1.35) 

±(1.99,2.45,2.88) 

(-2.07,0.44,3.76) 

±(2.25,1.59,2.69) 

(-1.26,1.76,0.87) 

±(1.99,2.97,4.01) 

table  VII 

Intraoperative  to  Postimplant  Movement  Direction  Results. 
Values  With  a  Mean  Value  to  Standard  Deviation  Ratio 
Greater  Than  0.95  are  in  Boldface 


Average  movement  direction  vectors  ( x,y,z )  (mm) 

left 

mid  y 

right 

ant 

(0.71,0.57,-0.38) 

±(1.99,1.16,2.37) 

(1.20,-0.00,-0.09) 

±(1.56,0.85,3.83) 

(0.29,-0.27,-0.41) 

±(1.91,0.77,3.03) 

inf 

mid  x 

(0.38,0.48,0.18) 

±(2.39,1.13,2.19) 

(1.75,-0.05,0.12) 

±(2.18,0.98,1.35) 

(0.39,-0.40,0.93) 

±(2.00,1.29,2.81) 

post 

(-0.01,1.09,0.36) 

±(1.40,1.15,2.31) 

(0.66,0.04,0.86) 

±(2.05,0.68,1.89) 

(0.08,-1.67,1.04) 

±(1.73,1.44,2.30) 

ant 

(-0.38,-0.12,-1.13) 

±(1.31,1.06,2.91) 

(0.70,-0.06,-1.29) 

±(1.26,0.74,3.59) 

(0.10,0.29,-1.12) 

±(1.31,0.98,3.27) 

mid  z 

mid  x 

(-1.12,0.72,-0.34) 

±(1.80,1.53,2.42) 

(-0.02,0.15,-0.66) 

±(0.46,0.74,1.57) 

(-0.72,-0.33,0.68) 

±(1.01,0.95,2.72) 

post 

(-0.79,0.97,0.98) 

±(1.12,1.67,2.37) 

(-0.59,0.03,1.36) 

±(0.99,0.75,1.85) 

(-0.66,-1.39,0.97) 

±(1.74,1.35,1.82) 

ant 

(0.53,-0.35,-1.65) 

±(1.25,1.24,3.22) 

(1.11,0.08,-0.63) 

±(1.11,0.99,2.78) 

(1.34,0.59,-0.50) 

±(1.33,1.31,3.85) 

sup 

mid  x 

(-0.69,0.11,-0.98) 

±(2.07,1.26,2.32) 

(-0.72,0.07,-1.03) 

±(1.41,0.94,1.26) 

(-0.11,-0.15,0.10) 

±(1.63,1.14,2.36) 

post 

(-0.08,0.49,0.14) 

±(2.01,1.40,2.32) 

(-0.34,-0.05,0.23) 

±(1.42,0.93,1.99) 

(-0.48,-1.00,0.41) 

±(1.97,1.46,1.75) 

As  with  the  scalar  measurements,  the  directional  displace¬ 
ments  from  intraoperative  to  postimplant  were  smaller  than  in 
the  preplan  to  intraoperative  case.  The  significant  intraoperative 
misplacement  results  can  be  summarized  as  follows:  1)  inferior 
displacement  of  lateral  anterior  seeds,  2)  superior  displacement 
of  medial  posterior  seeds,  and  3)  anterior  misplacement  of  su¬ 
perior  anterior  seeds.  For  the  postimplant  movement  there  is:  1) 
inward  lateral  movement  of  inferior  posterior  seeds  and  2)  an¬ 
terior  movement  of  superior  anterior  seeds. 

V.  Summary  and  Conclusion 

We  have  combined  several  techniques  to  formulate  a  new 
method  to  identify  seeds  from  stranded  implants  in  different 
datasets  and  therefore  compute  their  displacements.  The  com¬ 
plete  process  involved:  implantation  axis  detection,  needle  track 
detection  and  needle  matching  using  the  insertion  plane  and  the 


Intra-operative  misplacement  direction  vectors 


2  (1  mm/unit)  y  (1  mm/unit) 

Standard  deviation  ellipsoids 


z  (1  m m/unit)  y  (1  mm/unit) 


Fig.  13.  Intraoperative  misplacement  vectors  (above)  and  the  standard  devia¬ 
tion  ellipsoids  (below). 


Post-implant  movement  direction  vectors 


z  (0.5  mm/unit)  y  (0.5  mm/unit) 

Standard  deviation  ellipsoids 


z  (0.5  mm/unit)  y  (0.5  mm/unit) 

Fig.  14.  Postimplant  movement  vectors  (above)  and  the  standard  deviation  el¬ 
lipsoids  (below).  The  units  in  brackets  correspond  to  the  vector  lengths  and  not 
the  starting  positions.  Each  unit  is  1  mm  for  these. 

number  of  seeds  per  needle.  Our  iterative  best  line  detection  has 
accurately  found  the  implantation  axis  from  rotated  data  cluster 
in  all  the  datasets.  With  the  detected  implantation  axis,  seed  as¬ 
signment  based  on  a  minimum  cost  network  flow  algorithm  has 
an  accuracy  of  99.3%  and  99.4%  in  the  intraoperative  and  post¬ 
operative  data. 

Given  our  accurate  reconstructions,  we  were  able  to  measure 
the  magnitude  and  direction  of  seed  displacements.  For  the  re¬ 
gional  displacements,  the  following  conclusions  are  potential 
explanations  for  the  results  seen. 

For  the  preplan  to  intraoperative  comparison,  several  regional 
displacements  were  noted.  The  larger  amount  of  misplacement 
for  seeds  near  the  medial  line  is  most  likely  due  to  the  fact 
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that  this  is  the  longest  part  of  the  prostate  giving  the  oncolo¬ 
gist  more  leeway  to  steer  the  needles.  There  is  no  directional 
trend  to  this  placement  which  suggests  that  it  is  not  due  to  needle 
or  prostate  movement.  Similarly,  the  inferior  misplacement  of 
anterior  seeds  and  superior  misplacement  of  posterior  seeds  is 
also  likely  due  to  oncologist  tendencies.  There  is  a  lack  of  im¬ 
plantable  tissue  in  the  anterior  superior  quadrant  (close  to  the 
bladder)  and  so  seeds  are  deliberately  placed  more  inferiorly. 
The  greater  retraction  of  lateral  anterior  seeds  is  due  to  the  pres¬ 
ence  of  the  pubic  arch  which  forces  a  shallower  implant.  The 
divergence  of  the  rectum  from  the  prostate  in  the  superior  pos¬ 
terior  quadrant  leads  to  a  tendency  to  “over-plan”  the  medial 
superior  region  on  the  posterior  side. 

The  anterior  misplacement  of  superior  anterior  seeds  can  be 
explained  either  by  prostate  rotation  or  needle  deflection.  Any 
transverse  displacement  of  the  seeds  could  be  due  to  needle  de¬ 
flection.  We  expect  that  this  effect  would  be  most  visible  with 
superior  seeds.  However,  this  does  not  explain  the  anterior  di¬ 
rection  of  the  misplacement.  Therefore,  if  we  assume  that  it  is 
due  to  prostate  movement,  we  can  conclude  that  the  base  tends 
to  rotate  posteriorly  away  from  the  pubic  arch  during  insertion. 
This  leads  to  anterior  displacement  as  the  gland  rotates  back 
after  insertion.  We  will  examine  the  validity  of  these  explana¬ 
tions  in  our  future  studies. 

For  the  intraoperative  to  postimplant  comparison  the  first  di¬ 
rectional  conclusion  can  be  drawn  from  the  lack  of  a  global  out¬ 
ward  seed  displacement.  This  suggests  that  inflammation  has 
little  or  no  effect  for  immediate  postimplant  seed  movement. 
The  only  outward  motion  is  seen  with  anterior  movement  of  su¬ 
perior  anterior  seeds.  This  could  be  due  to  pressure  from  the 
bladder  on  the  superior  side  of  the  gland  when  the  patient  pose 
changes  from  dorsal  lithotomy  horizontal  recumbent.  Inward 
lateral  movement  of  inferior  posterior  seeds  must  also  be  due 
to  change  in  patient  pose  although  further  analysis  of  the  forces 
on  the  gland  during  the  change  is  needed  to  verify  this. 

Our  results  show  that  intraoperative  seed  misplacement  is 
larger  than  postimplant  movement.  This  confirms  results  from 
Chng  et  al  [9],  who  explain  the  large  impact  that  both  prostate 
rotation  and  needle  deflection  have.  This  also  agrees  with  work 
done  by  Wan  et  al  to  evaluate  needle  deflection  [23]  and  by 
Lagerburg  et  al  who  evaluate  prostate  rotation  during  the  in¬ 
sertion  of  needles  [24].  Furthermore,  our  technique  could  be 
used  to  track  seed  misplacement,  therefore  providing  a  measure¬ 
ment-based  alternative  to  studying  the  sensitivity  of  delivered 
dose  to  seed  misplacement  errors.  Su  et  al  performed  a  study 
on  the  effect  of  seed  misplacement  on  the  delivered  dose  using 
random  noise  to  model  the  misplacement  instead  of  actual  mea¬ 
surements  [25]  and  found  minimal  changes  in  the  D90  measure 
with  misplacements  of  up  to  4  mm.  However,  the  random  noise 
model  does  not  include  the  trends  we  found  and  so  the  seed  iden¬ 
tification  methods  used  here  could  be  used  to  confirm  this  result 
in  a  further  study. 

We  have  presented  regional,  directional  displacement 
measurement  techniques  for  stranded  seeds  in  prostate 
brachytherapy  which  can  be  used  both  to  influence  future 
planned  treatment  and  to  provide  more  post  surgery  informa¬ 
tion  to  patients  and  oncologists  for  quality  assurance. 


The  algorithms  explained  in  this  paper  have  been  described 
for  preplan  to  intraoperative  fluoroscopic  to  postimplant 
CT-data.  However,  they  can  also  be  used  to  compare  seed 
positions  over  several  days  after  a  surgery  to  further  monitor 
inflammation.  Another  application  of  the  developed  algo¬ 
rithms  is  in  real-time  dosimetry.  Complete  ultrasound-based 
brachytherapy  seed  detection  is  a  notoriously  challenging 
problem.  However,  Moradi  et  al  have  shown  that  given  a 
complete  reconstruction  of  the  fluoroscopic  data  and  identi¬ 
fication  of  the  seeds,  registration  can  be  performed  with  an 
incomplete  reconstruction  of  the  seed  cloud  in  ultrasound  data 
[13].  Therefore,  the  complete  seed  identification  (or  labeling) 
algorithm  described  here,  could  make  it  possible  to  perform 
intraoperative  dosimetry. 
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C-arm  fluoroscopy  images  are  frequently  used  for  qualitative  assessment  of  prostate  brachytherapy. 
Three-dimensional  seed  reconstruction  from  C-arm  images  is  necessary  for  intraoperative  dosimetry 
and  quantitative  assessment.  Seed  reconstruction  requires  accurately  known  C-arm  poses.  We  propose 
to  measure  the  C-arm  rotation  angles  and  computationally  compensate  for  inevitable  C-arm  motion  to 
compute  the  pose.  We  compensate  the  translational  motions  of  a  C-arm,  such  as  oscillation,  sagging 
and  wheel  motion  using  a  three-level  optimization  algorithm  and  obviate  the  need  for  full  pose  tracking 
using  external  trackers  or  fiducials.  We  validated  our  approach  on  simulated  and  100  clinical  data  sets 
from  10  patients  and  gained  on  average,  a  seed  matching  rate  of  98.5%,  projection  error  of  0.33  mm 
(STD  =  0.21  mm)  and  computation  time  of  19.8  s  per  patient,  which  must  be  considered  as  clinically 
excellent  results.  We  also  show  that  without  motion  compensation  the  reconstruction  is  likely  to  fail. 

©  201 1  Elsevier  B.V.  All  rights  reserved. 


1.  Introduction 

Prostate  cancer  continues  to  be  the  leading  cancer  among  men 
in  the  United  States  with  an  estimated  occurrence  of  217,730  new 
cases  in  2010  (Jemal  et  al.,  2010).  Low  dose  rate  brachytherapy 
(henceforth,  brachytherapy)  is  an  effective  treatment  for  localized 
prostate  cancer  that  can  achieve  excellent  outcomes  (Blasko  et  al., 
2002;  Morris  et  al.,  2009a, b).  Brachytherapy  entails  permanent 
placement  of  radioactive  capsules  (seeds)  of  125I  or  103Pd  inside 
the  prostate  and  periprostatic  tissue  to  kill  the  cancer  with  radia¬ 
tion.  Brachytherapy  seeds  are  as  small  as  a  grain  of  rice,  approxi¬ 
mately  4.5  mm  long  and  0.8  mm  in  diameter.  The  success  of  the 
procedure  directly  depends  on  the  accuracy  of  seed  placement  to 
deliver  sufficient  dose  to  eradicate  the  cancer  while  sparing  the 
urethra  and  rectum.  The  seed  positions  are  preplanned  to  tailor 
the  dose  to  the  patient’s  anatomy.  Generally  40-130  seeds  are  im¬ 
planted  depending  on  the  type  of  the  seeds  and  the  volume  of  the 
prostate.  During  the  procedure,  the  physician  delivers  the  seeds 
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using  needles  that  pass  through  a  guiding  template,  under  real¬ 
time  visual  guidance  from  transrectal  ultrasound  (TRUS)  (Prestidge 
et  al.,  1998).  The  guiding  template  confines  the  needles  to  move  in 
parallel  to  the  long  axis  of  the  probe.  C-arm  fluoroscopy  images  are 
frequently  taken  during  the  procedure  to  qualitatively  assess  the 
implant  (see  Fig.  1). 

Accurate  execution  of  the  plan  is  extremely  difficult  due  to 
prostate  displacement  and  deformation  (Lagerburg  et  al.,  2005), 
needle  bending,  prostate  swelling  (Yamada  et  al.,  2003)  and,  sys¬ 
tem  calibration  and  human  errors.  As  a  result,  seed  misplacements 
are  still  common  and  may  lead  to  under-dosed  regions  that  neces¬ 
sitate  repeated  treatment,  or  over-dosed  regions  which  result  in 
complications,  such  as  rectal  ulceration,  urinary  incontinence  and 
sexual  dysfunction.  Intraoperative  dosimetry  and  planning  meth¬ 
ods  were  introduced  to  reduce  the  errors  and  increase  the  treat¬ 
ment  quality  (Nag  et  al.,  2001;  Orio  et  al.,  2007;  Polo  et  al., 
2010).  These  methods  intermittently  calculate  the  delivered  dose 
during  the  procedure  and  modify  the  insertion  plan,  in  real-time, 
to  compensate  for  possible  errors. 

In  order  to  calculate  the  dose  distribution,  the  position  of  the 
implanted  seeds,  registered  to  the  prostate  anatomy,  should  be 
known.  Ultrasound  imaging  provides  sufficient  soft  tissue  contrast 
to  delineate  the  prostate;  however,  despite  the  efforts  (Han  et  al., 
2003;  Holmes  and  Robb,  2004;  Feleppa  et  al.,  2002;  McAleavey 
et  al.,  2003;  Mitri  et  al.,  2004;  Ding  et  al.,  2006;  Wei  et  al.,  2006; 
Wen  et  al.,  2010),  robust  seed  segmentation  in  ultrasound  is  not 
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Fig.  1.  Brachytherapy  procedure.  The  needle  passes  through  the  guiding  template 
in  parallel  to  the  long  axis  of  the  transrectal  probe.  The  C-arm  rotates  around  the 
craniocaudal  axis  of  the  patient  to  acquire  images.  This  figure  is  reproduced  from 
Lee  et  al.  (2009)  with  permission  form  the  authors. 


yet  possible  (up  to  25%  of  seeds  can  be  hidden  even  after  careful 
manual  segmentation  in  B-mode  images  (Han  et  al.,  2003)).  X- 
ray  fluoroscopy  images  provide  excellent  seed  visualization.  How¬ 
ever,  fluoroscopy  images  do  not  provide  sufficient  soft  tissue  con¬ 
trast  to  segment  the  prostate  boundaries.  Fluoroscopy-ultrasound 
fusion  can  provide  a  solution  for  real-time  intraoperative  dosime¬ 
try  and  has  been  extensively  studied  (French  et  al.,  2005;  Su  et  al., 
2006;  Orio  et  al.,  2007;  Tutar  et  al.,  2008;  Fallavollita  et  al.,  2010). 
In  this  scenario,  during  and/or  immediately  after  the  brachyther¬ 
apy  procedure,  several  C-arm  images  are  taken  from  different  an¬ 
gles  and  segmented  for  seed  centroids.  Then,  the  seeds  are 
localized  in  3D  space  using  the  segmented  images.  By  registration 
of  these  seeds  to  the  prostate  anatomy  localized  in  ultrasound,  the 
delivered  dose  to  the  prostate  can  be  calculated.  Upon  observation 
of  large  seed  misplacements  or  cold  spots,  the  physician  can  mod¬ 
ify  the  plan  and  implant  new  seeds.  Reconstruction  accuracy  and 
speed  are  vital  for  intraoperative  dosimetry  using  fluoroscopy- 
ultrasound  fusion. 

Three-dimensional  seed  reconstruction  has  been  widely  investi¬ 
gated  in  the  literature  (Amols  and  Rosen,  1981 ;  Tubic  et  al.,  2001a; 
Todor  et  al.,  2002;  Tutar  et  al.,  2003;  Su  et  al.,  2004;  Narayanan 
et  al.,  2004;  Lam  et  al.,  2004;  Jain  et  al.,  2005b;  Kon  et  al.,  2006; 
Brunet-Benkhoucha  et  al.,  2009;  Lee  et  al.,  2009;  Lee  et  al.,  2011). 
In  order  to  successfully  reconstruct  the  seed  positions  in  3D,  three 
major  problems  must  be  solved. 

(1)  C-arm  calibration :  For  an  accurate  reconstruction,  C-arm 
intrinsic  parameters,  such  as  image  resolution,  image  center, 
source  to  center  distance  and  focal  length  should  be  known 
(Navab  et  al.,  1996;  Brack  et  al.,  1996;  Jain  et  al.,  2005a). 

(2)  Seed  matching :  After  segmentation  of  the  2D  coordinates  of 
seed  projections  in  the  C-arm  images,  a  seed  matching  prob¬ 
lem  should  be  solved  to  assign  each  seed  projection  in  one 
image  to  the  corresponding  seed  projections  in  the  other 
images  (see  Fig.  2  for  an  example).  Since  the  seed  matching 
is  not  known,  seed  reconstruction  using  two  C-arm  images 
leads  to  an  ambiguity  that  can  be  resolved  using  a  third 
image.  Therefore,  at  least  three  images  are  required  for  seed 
localization  in  3D  space.  Seed  matching  has  been  solved 
using  various  methods,  such  as  simulated  annealing  (Tubic 
et  al.,  2001a),  heuristic  rules  (Todor  et  al.,  2002)  and  the 
Hungarian  algorithm  (Jain  et  al.,  2005b).  Hidden  or  overlap¬ 


ping  seed  projections  are  common  in  projection  images  and 
result  in  incomplete  data  sets  that  further  complicate  the 
seed  matching  problem  (see  Fig.  2).  The  hidden  seed  prob¬ 
lem  has  been  tackled  using  different  approaches,  such  as 
pseudo-seed-matching  (Narayanan  et  al.,  2004),  adaptive 
grouping  (Su  et  al.,  2004),  Hough  trajectories  (Lam  et  al., 
2004),  an  extension  to  the  Hungarian  algorithm  using  net¬ 
work  flow  (Kon  et  al.,  2006)  and  dimensionality  reduced  lin¬ 
ear  programing  (Lee  et  al.,  2011).  It  should  be  noted  that 
tomosynthesis-based  reconstruction  methods  have  been 
proposed  that  solve  the  matching  problem  automatically 
(Tutar  et  al.,  2003;  Brunet-Benkhoucha  et  al.,  2009;  Lee 
et  al.,  2009).  However,  these  methods  generally  need  a  larger 
number  of  images. 

Jain  et  al.  showed  that  seed  matching  is  equivalent  to  a  net¬ 
work  flow  problem  which  is  NP-Hard  (Jain  et  al.,  2005b). 
However,  they  proposed  a  pseudo-polynomial  yet  practical 
solution  for  seed  matching  from  three  images  by  mapping 
the  original  tripartite  problem  into  three  bipartite  ones  that 
could  be  solved  using  the  Hungarian  algorithm.  Jain’s 
method  was  abbreviated  as  MARSHAL  (Jain  et  al.,  2005b). 
The  original  MARSHAL  assumed  complete  data  sets;  how¬ 
ever,  it  was  later  extended  to  address  the  hidden  seed  prob¬ 
lem  (Kon  et  al.,  2006)  (the  extended  method  was 
abbreviated  as  XMARSHAL).  This  algorithm  demonstrated 
clinically  acceptable  reconstruction  rates  and  time  perfor¬ 
mance  on  simulated  and  phantom  data,  and  has  been  clini¬ 
cally  tested  (Song  et  al.,  2011;  Jain  et  al.,  in  press).  We  will 
discuss  XMARSHAL  in  more  details  in  Section  2.1  as  it  runs 
in  the  core  of  our  algorithm. 

(3)  Pose  recovery :  Seed  matching  and  reconstruction  are  per¬ 
formed  using  known  C-arm  poses  that  provide  the  relative 
positions  of  the  C-arm  images  in  3D  space.  The  C-arm  pose 
is  generally  recovered  using  radio-opaque  fiducials  or  beads 
(Navab  et  al.,  1996;  Bracket  al.,  1996;  Zhang  et  al.,  2004;  Jain 
et  al.,  2005a),  or  obtained  from  electromagnetic  and  optical 
trackers  (Peters  and  Cleary,  2008).  Fiducials  may  interfere 
with  the  image  of  the  anatomy,  require  segmentation  and 
can  limit  the  working  volume.  Auxiliary  trackers  are  expen¬ 
sive,  need  calibration,  optical  trackers  require  line  of  sight, 
electromagnetic  trackers  are  sensitive  to  interference  and 
hence,  further  complicate  the  intervention.  It  has  been  sug¬ 
gested  to  use  the  implanted  seeds  as  fiducials  to  compensate 
for  C-arm  pose  computation  errors  (Tubic  et  al.,  2001a;  Jain 
and  Fichtinger,  2006;  Lee  et  al.,  2009).  However,  a  good  ini¬ 
tial  measurement  of  the  pose  is  required  and  is  usually 
obtained  by  using  fiducials  and  trackers.  Pose  recovery  with¬ 
out  a  fiducial  or  tracker  can  significantly  ease  the  recon¬ 
struction  process  and  consequently  facilitate  the  transition 
of  seed  reconstruction  algorithms  from  research  laboratories 
to  medical  practice.  In  this  work,  we  introduce  a  method  to 
estimate  the  pose  using  sole  measurement  of  rotation  angles 
and  computationally  compensate  for  the  pose  computation 
errors  without  fiducials  or  external  trackers. 

It  is  common  practice  in  brachytherapy  to  acquire  several 
images  by  rotating  the  C-arm  around  the  patient.  Usually,  the  rota¬ 
tion  axis  is  approximately  aligned  with  the  patient’s  craniocaudal 
axis.  In  ideal  cases,  joint  angle  measurements  can  yield  an  accurate 
pose  of  the  C-arm.  However,  C-arm  movements,  such  as  oscillation, 
sagging  and  wheel  motion  are  significant  and  prevent  accurate 
pose  recovery  based  solely  on  joint  angle  measurements  (see 
Fig.  3).  If  uncompensated,  these  C-arm  motions  can  lead  to  severe 
pose  computation  errors  and  reconstruction  failure. 

In  this  paper,  we  prove  that  in  the  typical  case  of  a  C-arm  ro¬ 
tated  around  a  single  axis  within  a  small  angle  span,  C-arm  angle 
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Fig.  2.  Three  C-arm  images  taken  at  different  angles.  The  figure  at  the  middle  shows  several  hidden  and  overlapping  seeds.  Three  seed  matchings  are  shown  using  arrows.  The 
seeds  move  along  an  almost  horizontal  line  through  the  images. 


Fig.  3.  Initial  (gray)  and  correct  C-arm  pose  (color)  for  the  third  C-arm.  Transla¬ 
tional  error  along  the  up-down  direction  is  only  added  to  C-arm  3.  The 
reconstructed  seeds  are  shown  after  motion  compensation.  (For  interpretation  of 
the  references  to  color  in  this  figure  legend,  the  reader  is  referred  to  the  web  version 
of  this  article.) 

measurements  augmented  with  a  computational  method  to  com¬ 
pensate  for  C-arm  motions  suffice  for  a  clinically  reliable  and  accu¬ 
rate  seed  reconstruction.  However,  without  such  a  motion 
compensation  algorithm,  the  reconstruction  is  prone  to  failure. 
We  employ  joint  encoders  or  digital  protractors  to  measure  the 
rotation  angles.  The  novelty  of  this  work  is  in  the  introduction  of 
an  effective  motion  compensation  method  that  obviates  the  need 
for  full  pose  tracking  using  external  fiducials  or  trackers.  Consider¬ 
ing  the  simplicity  of  the  implementation,  high  reconstruction  accu¬ 
racy  and  favorable  computational  speed,  this  algorithm  is  suitable 
for  clinical  translation.  The  underlying  idea  and  limited  prelimin¬ 
ary  data  was  presented  at  a  recent  conference  (Dehghan  et  al., 
2010).  This  manuscript  provides  a  more  detailed  description  of 
the  methodology  and  performance  analysis  on  100  clinical  data 
sets. 

This  paper  is  organized  as  follows.  The  algorithms  for  seed 
matching  and  motion  compensation  are  explained  in  Section  2. 
Numerical  simulation,  phantom  and  clinical  results  are  presented 
in  Section  3,  followed  by  discussion  in  Section  4.  The  conclusions 
are  drawn  and  the  future  work  is  outlined  in  Section  5. 


2.  Methods 

We  propose  an  iterative  three-level  algorithm  that  takes  advan¬ 
tage  of  the  constrained  movement  of  a  C-arm  during  coplanar 
imaging,  compensates  for  its  major  translational  motions  using 
reconstructed  seeds  and,  in  turn,  significantly  increases  the  likeli¬ 
hood  of  finding  the  correct  matching  solution  in  a  clinically  accept¬ 
able  time.  We  compensate  for  C-arm  motions  that  are  mostly 
translational.  That  is  an  approximation  of  the  C-arm  motion  pat¬ 
tern.  However,  as  our  results  prove,  it  is  sufficient  for  successful 
reconstructions.  Our  motion  compensation  algorithm  can  be  used 
with  any  seed  matching  algorithm.  For  seed  matching  and  recon¬ 
struction,  we  employ  XMARSHAL  (Kon  et  al.,  2006)  that  is  capable 
of  solving  the  matching  problem  in  the  presence  of  hidden  seeds 
with  low  computational  cost.  For  the  sake  of  completeness  we 
briefly  outline  this  method  in  Section  2.1.  As  mentioned,  tomosyn¬ 
thesis-based  reconstruction  methods  do  not  require  a  matching 
algorithm.  However,  they  suffer  from  lack  of  accurate  C-arm  pose 
too.  Therefore,  C-arm  motion  compensation  framework  applies  to 
them  as  well. 

2  A.  Seed  reconstruction  using  XMARSHAL 

Assume  that  three  C-arm  images  of  an  implant  with  N  seeds  are 
available  and  Nit  i  e  {1,2,3}  seed  projections  are  segmented  in  each 
image.  In  this  case,  the  seed  matching  problem  in  the  presence  of 
hidden  seeds  can  be  written  as  the  following  optimization 
problem: 

N]  N2  N3 

mw  =  arS  min  X)  Y)  cijkmm,  (1 ) 

m«‘  i=i  j= i  k=\ 

'  n2  n3 

E  E  mrn  >  1 '  Vi 

j=  1  k= 1 

N-i  N3 

EE™p>  1,  Vj 

i= 1  k= 1 
S.t.  N1  N2 

ZZmm>  1,  Vk 

i=  1  j=  1 

Ni  N2  N3 

EE  E  mijk  =  N, 

i=  1  j=  1  k= 1 

k  mijk  e  {0,1}, 

where  for  each  i  e  {1,. .  .,Ni},  j  e  {1,. .  .,N2}  and  k  e  {1,. .  .,N3},  cijk  is 
the  cost  of  matching  seed  projections  p\ ,  pj  and  pi  from  the  first, 
second  and  third  images,  respectively,  and  mijk  is  a  binary  variable 
showing  the  correctness  of  such  a  match.  The  constraints  in  (1)  en¬ 
sure  reconstruction  of  N  seeds  while  taking  the  hidden  seeds  into 
account. 
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Assume  three  rays  emanating  from  each  of  these  seed  projec¬ 
tions  pj ,  pj  and  p3k  to  its  corresponding  X-ray  source  in  3D  space. 
The  point  with  the  minimum  average  distance  from  these  lines  is 
considered  as  their  symbolic  intersection.  We  define  the  average 
distance  of  the  symbolic  intersection  from  these  lines  as  their  sym¬ 
bolic  distance.  If  we  project  the  symbolic  intersection  on  each  im¬ 
age,  the  summation  of  the  distances  between  this  projection  and 
the  corresponding  seed  projection  on  each  image  is  considered  as 
the  matching  cost  (cp)  (Jain  et  al.,  2005b). 

Eq.  (1)  is  a  weighted-tripartite  matching  problem  that  is  equiv¬ 
alent  to  an  NP-Hard  combinatorial  optimization  (Jain  et  al.,  2005b) 
with  an  exponential  complexity.  An  approximate  low  computa¬ 
tional  cost  solution  to  this  problem  was  proposed  by  Jain  et  al. 
by  projecting  the  tripartite  matching  problem  into  three  bipartite 
problems  (Jain  et  al.,  2005b).  In  this  solution,  seed  projections  p\ 
and  pj  can  be  matched  (with  a  cost  q,)  only  if  they  have  a  low  cost 
counterpart  in  the  third  image.  In  this  method: 

Cij  =  min  c,j*,Vk.  (2) 

This  is  based  on  the  observation  that  although  low  cij}  cjk  and  cki  do 
not  guarantee  a  low  cijJo  a  low  cijk  guarantees  cijt  cjk  and  cki  to  be  low. 

This  solution  resembles  a  network-flow  optimization  problem 
in  which  each  seed  projection  in  an  image  is  represented  by  a  node 
(Jain  et  al.,  2005b;  Kon  et  al.,  2006).  The  matching  between  two 
seed  projections  is  represented  by  a  link  that  flows  between  them 
with  a  cost  equal  to  the  matching  cost  Cy.  This  network  flow  prob¬ 
lem  was  extended  in  Kon  et  al.  (2006)  to  address  the  hidden  seeds 
problem.  This  problem  can  be  solved  using  a  cycle  canceling  algo¬ 
rithm  in  practically  0(N3)  times  (Kon  et  al.,  2006),  producing  clin¬ 
ically  excellent  matching. 

Given  the  correct  matching  and  the  C-arm  poses,  the  3D  posi¬ 
tion  of  the  seeds  with  minimized  reconstruction  cost  is  calculated 
as: 


i£{ 


where  s,-  is  the  position  of  the  ith  seed,  q,  is  the  position  of  the  X-ray 
source  corresponding  to  the  jth  image,  Vy  is  the  unit  vector  along  the 
lines  Ly  that  connect  the  projection  of  seed  i  on  image  j  to  qj  and  I  is 
a  3  x  3  identity  matrix.  The  reconstruction  cost  for  seed  i  is  defined 
as  the  symbolic  distance  between  lines  Ln ,  Li2  and  Li3. 

2.2.  Motion  compensation 

Let  us  assume  a  world  coordinate  system  Oxwywzw  centered  at 
the  center  of  rotation  of  the  C-arm  and  a  source  coordinate  system 
Oxsyszs  centered  at  the  X-ray  source  as  shown  in  Fig.  4.  The  pose  of 
the  C-arm  is  defined  by  a  transformation  matrix  STW  from  the  world 
to  the  source  coordinate  system  as: 


'O' 

- 

ii 
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.  0  1 

where  SRW  is  the  rotation  matrix  from  the  world  to  the  source  coor¬ 
dinate  frame,  /  is  the  distance  from  the  source  to  the  center  of  rota¬ 
tion  and  S  =  [Sx  dy  Sz]'  is  the  translational  motion  of  the  C-arm 
caused  by  oscillation,  sagging  and  wheel  motion  (see  Fig.  3).  We 
can  initialize  a  pose  computation  by  measuring  the  C-arm  rotation 
angles  -  which  define  SRW  -  and  setting  the  unknown  S  equal  to 
zero.  Assuming  S  =  0  causes  error  in  the  pose  computation  and  sub¬ 


Fig.  4.  A  C-arm  rotating  around  its  primary  axis  (PA)  in  a  single  plane.  The  angle 
around  the  secondary  axis  (SA)  is  fixed.  The  world  coordinate  system  Oxwywzw  is 
centered  at  the  center  of  rotation.  The  source  coordinate  system  OXsyszs  is  centered 
at  the  source. 

sequent  unsuccessful  seed  reconstruction.  Therefore,  we  should 
compensate  for  this  error  and  improve  our  pose  computation. 

An  iterative  scheme  can  be  implemented  where  the  recon¬ 
structed  seeds  with  given  C-arm  poses  are  used  to  improve  the 
pose  recovery  and  subsequently  enhance  the  seed  reconstruction 
results.  Such  iterative  schemes  were  suggested  in  the  literature 
to  compensate  for  the  rotational  and  translational  pose  errors  (6 
DOFs)  (Tubic  et  al.,  2001a;  Jain  and  Fichtinger,  2006).  It  is  known 
that  such  a  problem  can  be  solved  up  to  an  unknown  scale  (Jain 
and  Fichtinger,  2006).  This  means  that  the  reconstructed  seed 
cloud  can  arbitrarily  shrink  or  expand.  As  it  is  shown  in  Fig.  5, 
the  X-ray  images,  C-arm  intrinsic  parameters,  seed  matching  and 
relative  C-arm  rotations  are  identical  between  two  sets  of  recon¬ 
struction  solutions;  however,  the  scaled  relative  translations  result 
in  a  scaled  seed  cloud. 

We  demonstrate  that  by  making  realistic  and  practical  assump¬ 
tions  in  accordance  with  clinical  protocols,  motion  compensation 
can  be  reduced  to  2  DOFs  for  mobile  C-arms  used  in  brachytherapy. 
This  approach  eliminates  the  scaling  problem  and  results  in  a  clin¬ 
ically  adequate  implant  reconstruction  and  computational  time 
small  enough  to  be  carried  out  intraoperatively.  Pose  error  com- 


Fig.  5.  The  scaling  effect  results  from  3  DOF  translational  motion  of  the  C-arm.  The 
X-ray  images,  C-arm  relative  rotations  and  C-arm  intrinsic  parameters  are  identical 
between  the  left  and  right  reconstructions.  Seed  matching  solution  is  independent 
of  the  scaling  factor. 
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pensation  for  2  DOFs  requires  less  complicated  algorithms  (com¬ 
pare  to  6  DOFs)  and  thus  is  computationally  faster. 

We  make  the  following  assumptions: 

1.  The  C-arm  images  are  acquired  by  rotation  of  a  C-arm  around 
its  primary  axis  (PA)  in  a  small  angle  span,  while  the  angle 
around  the  secondary  axis  (SA)  is  fixed  (see  Fig.  4). 

2.  C-arm  rotation  angles  are  measured. 

3.  The  intrinsic  parameters  are  known  and  do  not  change  during 
rotation  of  the  C-arm. 

4.  2D  coordinates  of  seed  projections  are  available  via  manual  or 
automatic  segmentation. 

5.  Most  significant  C-arm  motions  are  of  translational  nature,  con¬ 
fined  to  the  Oywzw  plane  -  the  motion  along  xw  is  negligible 
(^ix  =  0,i  E  {1,2,3}). 


As  mentioned  before,  acquiring  images  by  rotating  the  C-arm 
around  its  PA  while  the  angle  around  SA  is  fixed  is  common  prac¬ 
tice  in  brachytherapy.  The  rotation  span  around  PA  is  usually  lim¬ 
ited  to  approximately  ±15°  due  to  collision  of  the  C-arm  with  the 
operating  table,  brachytherapy  stand  or  patient’s  legs.  Mobile  C- 
arms  are  available  that  employ  a  fixed  axis  of  rotation  and  are 
equipped  or  can  be  easily  retrofitted  with  joint  encoders  (Grzeda 
and  Fichtinger,  2010). 

Limited  rotation  span  in  a  clinical  setting  results  in  an  insignif¬ 
icant  change  in  the  calibration  parameters,  such  as  focal  length, 
image  center  and  source  to  center  distance.  Image  resolution  is 
considered  as  constant  over  the  lifetime  of  the  device.  It  has  been 
shown  that  in  such  situations  recalibration  for  each  C-arm  pose  is 
not  necessary,  because  small  changes  in  the  calibration  parameters 
do  not  significantly  alter  the  relative  positions  of  the  reconstructed 
seeds  (Jain  et  al.,  2007).  It  should  be  noted  that  we  are  interested  in 
the  relative  position  of  the  seeds  as  the  seed  cloud  as  a  whole 
should  later  be  registered  to  the  prostate  anatomy. 

In  this  work  we  relied  on  manual  segmentation  to  identify  the 
seed  centroids.  Since  XMARSHAL  is  capable  of  addressing  the  hid¬ 
den  seeds  problem,  it  is  not  required  to  identify  all  the  implanted 
but  hidden  seeds  in  every  image.  The  effects  of  hidden  seed  per¬ 
centage  and  seed  segmentation  error  on  the  performance  of  XMAR¬ 
SHAL  are  discussed  in  Kon  et  al.  (2006). 

The  last  assumption  is  the  cornerstone  of  our  motion  compen¬ 
sation  algorithm.  The  most  likely  source  of  pose  computation  error 
is  the  oscillation  of  the  C-arm,  which  is  mostly  up-down  (along  the 
zw  axis)  since  the  C-arm  is  connected  to  the  base  as  a  cantilever. 
Our  observations  confirm  this  assumption  that  the  C-arm  primary 
motion  is  in  the  up-down  direction  due  to  oscillation  and  C-arm 
weight.  The  motion  in  the  other  two  directions  are  much  smaller; 
however,  are  not  always  insignificant.  As  a  result,  C-arm  pose  com¬ 
putation  error  along  zw  is  more  significant  compared  to  the  inaccu¬ 
racies  in  the  other  two  directions. 

The  proposed  motion  compensation  algorithm  initializes  the  C- 
arm  pose  using  measured  joint  angles  as  shown  in  Fig.  3  and  com¬ 
pensates  for  the  inevitable  translational  motions  by  finding  the 
optimal  translational  adjustments  (offsets)  3  for  each  source  posi¬ 
tion,  by  solving  the  following  problem: 

N,  N2  N3 

(mjjk,  d*n)  =  argmin  Y)  cm(sn)miJk,  (5) 

V  7  Xi*'5n  i=  1  j=  1  k=  1 

n  g  {1,2, 3}, 


subject  to  the  constraints  of  (1).  In  order  to  minimize  the  scaling  ef¬ 
fects,  we  restrict  the  offsets  to  be  in  the  Oywzw  plane  -  based  on  our 
assumption  -  and  introduce  a  three-level  optimization  method  to 
identify  them. 


2.2.1.  First-level  optimization 

At  the  first  level,  the  algorithm  finds  a  few  matching  seed  pro¬ 
jections  in  the  images  to  calculate  an  initial  offset  estimate  with 
very  low  computational  cost.  Since  the  fluoroscopy  images  are  ta¬ 
ken  with  the  rotation  of  the  C-arm  around  its  PA,  the  seed  projec¬ 
tions  follow  almost  horizontal  lines  in  the  images.  Therefore,  a  seed 
at  the  top  or  bottom  of  one  image  is  more  likely  located  at  the  top 
or  bottom  of  the  other  images  (for  example  see  Fig.  2).  Based  on 
this  observation,  the  algorithm  automatically  selects  n  seed  projec¬ 
tions  from  the  top  (seeds  with  maximum  ordinate  values  in  the  2D 
image  coordinate  system)  and  n  seed  projections  from  the  bottom 
of  each  image  (seeds  with  minimum  ordinate  values  in  the  2D  im¬ 
age  coordinate  system)  and  solves  the  matching  problem  for  them, 
using  the  initial  estimates  of  the  C-arm  poses  from  joint  angle 
readings.  Since  the  selected  2 n  seed  projections  from  one  image 
do  not  necessarily  correspond  to  the  selected  2 n  seed  projections 
in  the  other  images  (these  6n  seed  projections  may  belong  to  more 
than  2 n  seeds  in  3D),  some  of  the  reconstructed  seeds  have  a  high 
reconstruction  cost,  are  erroneously  matched,  and  cannot  be  used 
to  improve  the  pose  recovery.  Therefore,  the  algorithm  selects  a 
subset  of  p<n  reconstructed  seeds  from  the  top  and  p  recon¬ 
structed  seeds  from  the  bottom  of  the  2 n  reconstructed  seeds  with 
the  best  reconstruction  costs  for  pose  recovery.  We  use  n  =  5  and 
p  =  2.  In  order  to  estimate  the  C-arm  motion,  we  assume  that  the 
position  of  the  C-arm  source  corresponding  to  the  first  image 
(henceforth,  the  first  source  position)  in  3D  space  is  fixed  (<5i  =  0) 
and  optimize  the  position  of  the  C-arm  sources  corresponding  to 
the  second  and  third  images  (henceforth,  the  second  and  third 
source  positions)  in  the  Oywzw  plane  to  minimize  the  reconstruc¬ 
tion  cost  for  the  selected  2 p  seeds.  The  seed  matching  and  motion 
estimation  are  iteratively  performed  until  there  is  no  change  in  the 
seed  matching  solution. 

Since  a  small  number  of  seed  projections  are  used  at  this  level, 
the  matching  problem  can  be  solved  extremely  quickly  using 
XMARSHAL  With  a  given  matching,  2D  offsets  for  the  second 
and  third  C-arm  source  positions  can  be  found  by  solving  a  set  of 
linear  equations.  These  equations  are  derived  in  (A.6)-(A.15). 


2.2.2.  Second-level  optimization 

In  this  step,  the  C-arm  source  positions  are  initialized  in  3D 
space  using  the  optimal  offset  values  from  the  first  level.  Then, 
the  matching  problem  is  solved  for  all  seed  projections  and  the 
seeds  are  reconstructed.  The  C-arm  source  positions  are  optimized 
in  3D  space  to  minimize  the  average  reconstruction  cost  while  the 
seeds  are  fixed  in  space.  The  matching  and  source  position  optimi¬ 
zations  are  iteratively  solved  until  the  reconstruction  cost  or  its 
change  between  two  iterations  is  smaller  than  a  predefined  thresh¬ 
old.  The  thresholds  were  empirically  assigned  as  <0.1  mm  for  the 
former  and  <0.1%  for  the  latter. 

The  new  position  of  the  jth  X-ray  source  at  iteration  k  +  1  is  cal¬ 
culated  as: 


qf+1  = 


N 


(6) 


where  s-  is  the  position  of  the  ith  seed  at  iteration  k. 

Since  we  employ  a  3D  optimization  at  this  level,  the  recon¬ 
structed  seed  cloud  may  significantly  shrink  or  expand  (see 
Fig.  5).  However,  optimization  in  3D  increases  the  likelihood  of 
finding  the  correct  matching  which  is  independent  of  the  scaling 
factor.  Therefore,  we  take  advantage  of  3D  optimization  at  this  le¬ 
vel,  find  the  correct  matching  and  remove  the  scaling  effects  at  the 
next  level. 
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2.2.3.  Third-level  optimization 

At  this  step,  we  assume  that  the  correct  matching  solution  is 
available  from  the  second-level  optimization.  However,  the  seed 
cloud  may  be  scaled  due  to  3D  motion  compensation.  Therefore, 
once  more,  we  assume  that  the  C-arm  motion  along  xw  is  negligible 
and  optimize  for  2D  C-arm  motion.  The  C-arm  source  positions  are 
initialized  using  the  joint  angle  readings.  The  first  source  is  fixed  in 
space.  Next,  2D  offsets  of  the  second  and  third  source  positions  are 
optimized  to  minimize  the  reconstruction  cost  with  the  given 
matching.  Similar  to  the  first  level,  the  3D  seed  positions  and  2D 
offsets  for  the  sources  have  a  closed  form  solution,  the  equations 
for  which  are  derived  in  (A.6)-(A.15).  The  three  steps  of  the  algo¬ 
rithm  are  shown  in  Fig.  6. 

3.  Results 

3.1.  Simulations 

First,  the  motion  compensation  algorithm  was  tested  on  simu¬ 
lated  data.  We  synthesized  four  seed  clouds  based  on  realistic 
dosimetry  plans  of  four  patients  with  100, 102, 108  and  130  seeds. 
Seed  images  were  synthesized  by  rotation  of  the  C-arm  around  the 
PA  at  0°,  ±5°  and  ±10°,  while  the  SA  angles  were  kept  constant  at 
180°.  The  intrinsic  parameters  of  a  GE  OEC®  9800  device  were  used 
as  the  intrinsic  parameters  of  the  C-arm  in  the  simulations.  The 
seeds  were  reconstructed  using  every  possible  combination  of 
three  images  out  of  five.  In  order  to  investigate  the  performance 
of  our  motion  compensation  algorithm,  translational  and  rota¬ 
tional  pose  errors  were  added  to  one  of  the  C-arm  source-image 
pairs.  The  added  errors  were  0-10  mm  along  xw  and  yw,  0- 
20  mm  along  zw  with  steps  of  1  mm  and  0-3°  around  SA  and  PA 
with  steps  of  0.5°.  The  effects  of  these  errors  were  simulated  inde¬ 
pendently,  as  only  one  error  was  introduced  at  each  simulation. 
During  image  synthesis,  hidden  seed  projections  were  created  by 
merging  the  seed  projections  that  were  close  to  each  other.  There 
were  on  average  1.6  hidden  seeds  per  image,  with  a  maximum  of 
14. 


The  reconstructed  seeds  were  compared  against  the  ground 
truth  after  a  rigid  registration  of  reconstructed  seed  cloud  to  the 
known  seed  cloud.  The  average  and  standard  deviation  (STD)  of 
localization  error,  defined  as  the  distance  between  the  true  and 
reconstructed  seeds,  are  shown  in  Fig.  7  for  all  the  introduced  pose 
errors.  The  average  and  STD  of  the  matching  rate  are  also  shown  in 
Fig.  7.  The  algorithm  has  an  average  matching  rate  of  99.2%  when 
the  pose  error  is  zero. 

As  it  can  be  seen  in  Fig.  7c-f,  the  algorithm  shows  consistently 
high  matching  rates  and  small  localization  errors  over  a  wide  range 
of  errors  along  yw  and  zw  as  the  C-arm  motions  in  these  two  direc¬ 
tions  are  compensated.  Fig.  7a  shows  consistently  high  matching 
rates  for  errors  of  up  to  approximately  5mm  along  xw.  However, 
the  localization  error  increases  monotonically  with  the  error  in  this 
direction  as  shown  in  Fig.  7b.  This  is  due  to  the  fact  that  the  match¬ 
ing  problem  is  solved  at  the  second  level,  using  a  3D  motion  com¬ 
pensation;  while  we  reconstruct  the  seeds  using  a  2D  motion 
compensation  at  the  third  level.  Therefore,  the  correct  matching 
is  found  at  the  second  level  even  in  the  presence  of  errors  along 
xw;  however,  the  seed  cloud  is  deformed  at  the  third  level,  which 
leads  to  a  monotonically  increasing  localization  error.  The  match¬ 
ing  and  localization  errors  increase  with  the  rotational  pose  errors, 
since  the  motion  compensation  algorithm  does  not  compensate  for 
rotational  errors.  However,  the  average  matching  rate  is  above  95% 
when  rotational  pose  errors  are  less  than  1.5°.  We  assume  that  in  a 
clinical  setting,  the  angles  are  measured  with  errors  of  less  than 
±1°. 

3.2.  Phantom  study 

We  conducted  a  phantom  study  on  a  CIRS  Model  053  tissue- 
equivalent  prostate  brachytherapy  training  phantom.  An  experi¬ 
enced  brachytherapist  inserted  26  needles  to  implant  136  dummy 
stranded  seeds  inside  the  phantom. 

We  took  five  images  from  the  phantom  using  a  GE  OEC®  9800 
mobile  C-arm  by  rotation  of  the  device  around  its  PA  in  a  20°  rota¬ 
tion  span  in  approximately  5°  intervals.  A  digital  protractor  was 
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Fig.  6.  The  proposed  three-level  motion  compensation  algorithm. 
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Fig.  7.  Simulation  results,  showing  the  average  and  standard  deviation  of  matching  rate  and  localization  error  for  variable  pose  errors.  The  average  and  STD  of  matching  rate 
for  errors  along  xw,  yw,  zw,  PA  and  SA  are  shown  in  (a),  (c),  (e),  (g)  and  (i),  respectively.  The  average  and  STD  of  localization  error  for  errors  along  xw,  yw,  zw,  PA  and  SA  are  shown 
in  (b),  (d),  (f),  (h)  and  (j),  respectively. 
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connected  to  the  source  casing  of  the  device  to  accurately  measure 
the  rotation  angles.  The  seed  projections  were  manually  seg¬ 
mented  in  the  images  and  the  seeds  were  reconstructed  in  3D 
using  every  combination  of  3  out  of  5  images  (10  reconstructions) 
using  motion  compensation. 

The  phantom  was  also  scanned  using  a  Picker  PQ5000  CT  scan¬ 
ner.  The  seeds  in  the  CT  volume  were  segmented  by  thresholding 
and  assumed  as  the  ground  truth.  Although  the  CT  and  C-arm 
images  were  taken  at  different  times,  we  assumed  that  the  phan¬ 
tom  deformation  and  seed  displacements  were  negligible.  We 
compared  the  seeds  reconstructed  from  C-arm  images  to  the  seeds 
from  CT  after  a  rigid  registration  of  the  two  seed  clouds  and  re¬ 
ported  the  difference  as  localization  error  in  Table  1. 

On  average,  we  achieved  a  matching  rate  of  99.0%  with  0.9  mm 
localization  error.  In  order  to  show  the  importance  of  motion  com- 
pensaion,  we  also  tried  to  reconstruct  the  seeds  without  motion 
compensation  and  achieved  an  average  of  78.3%  for  matching  rate, 
which  is  far  below  the  clinically  acceptable  level.  Su  et  al.  sug¬ 
gested  that  a  seed  detection  rate  of  above  95%  is  required  in  order 
to  achieve  clinically  sufficient  estimation  of  dose  distribution  for 
contemporary  125I  prostate  implants  (Su  et  al.,  2005). 


3.3.  Performance  on  clinical  data 

The  performance  of  the  motion  compensation  algorithm  was 
also  tested  on  clinical  data.  Ten  patients  were  implanted  with 
100-135  125I  seeds  (average  112)  at  the  British  Columbia  Cancer 
Agency.  Five  C-arm  images  were  taken  from  each  patient  by  rota¬ 
tion  of  the  C-arm  around  the  patient’s  craniocaudal  axis  at  angles 
approximately  0°,  ±5°  and  ±10°,  while  the  SA  angle  was  fixed.  A 
GE  OEC®  9800  mobile  C-arm  was  used  for  imaging.  This  is  a  digital 
device  with  motorized  joints.  This  device  has  a  heavy  intensifier 
that  causes  significant  sagging  and  oscillation  during  image  acqui¬ 
sition.  For  patients  1-8  the  rotation  angle  around  the  PA  was  mea¬ 
sured  by  a  digital  protractor  which  was  attached  to  the  source 
casing.  The  digital  protractor  did  not  interfere  with  the  image  of 
the  anatomy  or  the  working  space  and  did  not  require  precalibra¬ 
tion.  For  the  other  2  patients  the  rotation  angles  were  measured 
using  the  joint  encoders  of  the  device.  The  digital  protractor  had 
a  resolution  of  0.1°  while  the  device  joint  encoders  had  a  resolution 
of  1°.  We  expect  higher  accuracy  from  the  measurements  of  the 
protractor.  The  C-arm  joint  angle  encoders  showed  a  variation  of 
1°  around  the  SA.  This  deviation  was  taken  into  account  for  initial¬ 
ization  of  the  C-arm  poses.  The  C-arm  intrinsic  parameters  were 
once  identified  in  a  preoperative  calibration  and  were  assumed 
to  be  constant  for  all  the  rotation  angles  and  all  the  patients.  For 
each  patient,  we  reconstructed  the  seeds  for  every  combination 
of  three  images  out  of  available  five,  thus  obtaining  100  recon¬ 
structions  in  total.  The  seeds  were  manually  segmented  in  the 
images.  There  were  an  average  of  2  and  maximum  of  8  hidden 
seeds  in  the  images. 


Table  1 

Matching  rates  and  mean  and  standard  deviation  of  localization  error  for  phantom. 


Rec.  # 

Matching  rate  (%) 

Localization  error  mean  ±  STD  (mm) 

1 

100.0 

1.0  ±0.6 

2 

100.0 

0.9  ±  0.5 

3 

100.0 

0.7  ±  0.4 

4 

99.2 

0.9  ±  0.6 

5 

98.5 

0.8  ±  0.4 

6 

98.5 

0.8  ±  0.5 

7 

98.5 

1.0  ±0.6 

8 

98.5 

0.8  ±  0.7 

9 

99.2 

0.8  ±  0.4 

10 

97.8 

0.9  ±  0.5 

Fig.  8.  Reprojected  seeds  overlaid  on  a  C-arm  fluoroscopy  image,  showing  small 
projection  errors. 

The  reconstructed  seeds  were  reprojected  on  the  images  (see 
Fig.  8)  and  were  meticulously  inspected  for  matching  errors.  These 
images  were  also  used  to  measure  the  projection  error  -  defined  as 
the  distance  between  the  segmented  seed  centroid  and  the  pro¬ 
jected  location  of  the  reconstructed  seed  -  to  quantify  the  recon¬ 
struction  accuracy. 

Fig.  9a  shows  the  seed  matching  rate  with  motion  compensa¬ 
tion  for  each  case.  Overall,  we  achieved  an  average  matching  rate 
of  98.5%  which  is  above  the  clinically  accepted  level.  A  perfect 
reconstruction  was  accomplished  in  54%  of  the  cases,  while  in 
76%  and  92%  of  the  cases  the  matching  rate  was  greater  or  equal 
to  98%  and  95%,  respectively. 

In  order  to  demonstrate  the  necessity  of  motion  compensation, 
seed  reconstruction  was  performed  without  motion  compensation 
as  well.  As  it  can  be  seen  in  Fig.  9b,  XMARSHAL  without  motion 
compensation  achieved  an  average  matching  rate  of  46.1%,  which 
is  a  completely  inadequate  performance.  Moreover,  calculations 
show  that  motions  of  more  than  30  mm  in  the  up-down  direction 
were  compensated.  This  proves  our  hypothesis  that  it  is  necessary 
to  use  motion  compensation,  when  only  C-arm  rotation  angles  are 
measured. 

Fig.  9b  directs  us  to  another  important  role  of  the  first-level 
optimization  -  other  than  increasing  the  speed.  As  it  can  be  seen 
in  Fig.  9b,  an  initial  reconstruction  without  application  of  the 
first-level  optimization  can  result  in  very  unsuccessful  matching 
results  in  which  mismatched  seeds  are  significantly  abundant 
and  can  even  outnumber  the  correctly  matched  seeds.  In  such 
cases,  sole  application  of  the  second-level  optimization  can  result 
in  erroneous  pose  estimations  since  the  large  group  of  mismatched 
seeds  can  misguide  the  optimization  algorithm.  However,  in  the 
first-level  optimization,  the  algorithm  selects  a  handful  of  seeds 
from  the  top  and  bottom  of  the  images  that  are  very  likely  to  cor¬ 
rectly  match.  These  few  correctly  matched  seeds  provide  a  good 
initial  pose  computation  for  the  second  level.  Therefore,  the  first- 
level  optimization  not  only  shortens  the  computation  time  but  also 
increases  the  robustness  of  the  algorithm. 

The  mean  and  the  standard  deviation  of  the  projection  error  are 
shown  in  Fig.  9c  and  d  for  correctly  and  erroneously  matched 
seeds,  respectively.  The  overall  average  and  standard  deviation  of 
the  projection  error  for  correctly  matched  seeds  are  0.33  mm  and 
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(a)  Matching  rate  with  motion  compensation. 
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(b)  Matching  rate  without  motion  compensation. 
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(d)  Mean  and  STD  of  projection  error  for  mismatched 
seeds. 


Fig.  9.  Clinical  results  showing  the  matching  rate  (a)  with  and  (b)  without  motion  compensation,  the  mean  and  STD  of  the  projection  error  for  (c)  correctly  and  (d) 
erroneously  matched  seeds.  In  (c)  and  (d),  the  length  of  the  green  bars  shows  the  mean,  and  the  length  of  the  orange  bars  shows  the  STD. 


0.21  mm,  respectively,  which  demonstrates  high  reconstruction 
accuracy.  The  mean  and  standard  deviation  of  projection  errors 
are  1.12  mm  and  1.93  mm,  respectively,  for  mismatched  seeds. 

It  should  be  noted  that  the  projection  error  cannot  be  used  to 
measure  the  scaling  factor.  Therefore,  we  use  seed  spacing  to  pro¬ 
vide  additional  confidence  in  our  reconstruction  accuracy.  The  pa¬ 
tients  in  our  study  were  implanted  using  stranded  125I  seeds.  In 
such  a  case,  the  seeds  that  are  implanted  by  a  common  needle 
are  connected  to  each  other  by  a  bio-degradable  strand  that  keeps 
them  at  a  fixed  center-to-center  distance  of  10  mm.  We  measured 
the  center-to-center  distances  of  the  seeds  that  were  inserted 
using  a  common  needle  and  reported  the  results  for  each  patient 
in  Table  2.  The  overall  seed  spacing  was  10.3  mm,  which  shows 
that  significant  scaling  did  not  occur. 

4.  Discussion 

We  achieved  matching  rates  comparable  to  the  results  reported 
in  Lee  et  al.  (2009),  Brunet-Benkhoucha  et  al.  (2009),  Lee  et  al. 
(201 1 ).  In  particular,  similar  matching  rates  are  reported  in  the  ori¬ 
ginal  MARSHAL  and  XMARSHAL  papers  (Jain  et  al.,  2005b;  Kon 
et  al.,  2006).  However,  it  should  be  noted  that  in  all  these  works 
the  C-arm  pose  was  accurately  known  by  using  either  a  radio-opa¬ 
que  fiducial  (Jain  et  al.,  2005b;  Kon  et  al.,  2006;  Lee  et  al.,  2009;  Lee 
et  al.,  201 1 )  or  a  precisely  calibrated  and  accurately  tracked  radio¬ 
therapy  simulator  in  Brunet-Benkhoucha  et  al.  (2009 )(It  should  be 
noted  that  therapy  simulators  are  extinct  in  contemporary  radia¬ 
tion  oncology).  In  other  words,  the  C-arm  was  fully  tracked  and 
accurate  poses  were  available.  Due  to  availability  of  accurate  C- 
arm  poses  in  these  works,  motion  compensation  was  not  neces¬ 
sary.  However,  in  our  approach  no  external  tracker  was  used  to 
estimate  the  pose.  Therefore,  the  initial  pose  recovery  without  mo¬ 
tion  compensation  in  our  case  was  not  sufficiently  accurate  for  a 


Table  2 

The  average  and  standard  deviation  of  the  distance 
between  two  consecutive  seeds  implanted  by  one 
needle. 


Patient  # 

Seed  spacing  (mm)  Mean  ±  STD 

1 

10.4  ±0.6 

2 

10.4  ±0.6 

3 

10.3  ±0.4 

4 

10.3  ±0.3 

5 

10.3  ±0.3 

6 

10.2  ±0.4 

7 

10.3  ±0.4 

8 

10.1  ±0.4 

9 

10.4  ±0.5 

10 

10.3  ±0.4 

Overall 

10.3  ±0.5 

successful  reconstruction  (see  Fig.  9b).  However,  with  the  pro¬ 
posed  computational  motion  compensation  method  we  achieved 
high  matching  rates  without  full  pose  tracking  using  any  external 
tracker  or  fiducial  (see  Fig.  9a).  This,  in  particular,  explains  the  dis¬ 
crepancy  between  the  successful  results  reported  in  Jain  et  al. 
(2005b),  Kon  et  al.  (2006)  and  our  unsuccessful  results  with  the 
same  matching  algorithm  when  motion  compensation  was  not 
used. 

In  current  brachytherapy  practice,  implant  geometry  is  assessed 
using  CT,  one  or  several  days  after  the  procedure.  However,  the 
fluoroscopy  images  are  taken  at  the  end  of  the  procedure  while 
the  patient  is  still  in  treatment  position  and  additions  to  the  im¬ 
plant  are  still  possible  to  patch  up  cold  spots.  The  prostate  swells 
during  and  after  the  procedure  which  results  in  a  seed  displace¬ 
ment  over  time  (Yamada  et  al.,  2003)  and  even  during  the  proce¬ 
dure  (Jain  et  al.,  in  press).  The  seeds  also  tend  to  migrate  after 
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implantation  (Fuller  et  al.,  2004).  Moreover,  for  our  patients,  the 
fluoroscopy  images  were  taken  while  the  ultrasound  probe  was 
still  partially  inside  the  rectum.  The  probe  can  deform  the  prostate 
and  displace  the  seeds.  Therefore,  the  physical  position  of  the  seeds 
during  the  CT  scan  is  likely  to  be  different  from  the  position  of  the 
seeds  when  the  C-arm  images  are  taken.  Hence,  CT  images  of  the 
patient  can  not  be  used  as  a  confident  ground  truth  to  measure 
the  localization  error.  Therefore,  we  relied  on  the  projection  error 
to  validate  the  reconstruction  in  our  clinical  study.  For  the  same 
reason,  we  had  to  rely  on  reprojected  images  to  identify  the  mis¬ 
matches.  It  was  shown  that  seed  localization  errors  of  less  than 
2  mm  result  in  less  than  5%  deviation  in  the  prostate  D90  (the  min¬ 
imum  dose  delivered  to  90%  of  the  prostate  volume)  (Lindsay  et  al., 
2003;  Su  et  al.,  2007).  Although  it  was  not  possible  for  us  to  mea¬ 
sure  the  localization  error  in  our  clinical  study,  the  localization  er¬ 
rors  in  our  simulations  and  phantom  study  were  significantly 
lower  than  this  threshold. 

Although  the  mean  and  STD  of  the  projection  error  for  mis¬ 
matched  seeds  are  significantly  larger  than  those  of  the  correctly 
matched  seeds  (see  Fig.  9c  and  d),  the  range  of  the  projection  error 
for  correctly  and  erroneously  matched  seeds  overlap.  Therefore,  a 
fixed  threshold  for  the  projection  error  cannot  be  used  as  a  crite¬ 
rion  to  reliably  detect  the  mismatches.  However,  it  should  be  noted 
that,  in  some  cases,  a  seed  projection  is  mistakenly  matched  to  a 
seed  projection  which  is  located  very  close  to  the  correct  one.  In 
such  cases,  the  projection  error  is  small.  Such  mismatches  result 
in  small  errors  in  seed  localization.  This  suggests  that  if  the  seeds 
with  large  projection  errors  are  removed,  the  rest  of  the  mis¬ 
matches  may  result  only  in  an  insignificant  change  in  the  dosime¬ 
try.  Statistically,  only  17.3%  of  mismatched  seeds  had  a  projection 
error  larger  than  2  mm.  This  is  0.2%  of  all  the  reconstructed  seeds. 

We  used  three  C-arm  images  for  seed  reconstruction.  While 
more  could  have  been  used,  the  fewer  images  used  in  the  OR  the 
better,  primarily  because  it  saves  time  and  also  reduces  radiation 
exposure  to  OR  staff.  The  matching  rate  is  likely  to  improve  by 
using  more  than  three  images  at  the  expense  of  computational 
complexity  (Kon  et  al.,  2006).  The  motion  compensation  algorithm 
and  the  seed  matching  method  are  valid  for  an  arbitrary  number  of 
images.  Alternatively,  seeds  can  be  reconstructed  for  every  combi¬ 
nation  of  three  out  of  all  available  images  for  a  patient  and  the  best 
reconstruction  can  be  chosen.  The  reconstruction  algorithm  is  suf¬ 
ficiently  fast  to  allow  for  such  a  scheme  in  clinical  settings.  It 
should  be  noted  that  for  all  the  patients,  except  patient  3,  there 
is  at  least  one  reconstruction  with  perfect  matching  (see  Fig.  9a). 

The  motion  compensation  algorithm  can  be  combined  with 
other  seed  matching  algorithms,  such  as  REDMAPS  (Lee  et  al., 
2011)  as  reported  in  Dehghan  et  al.  (2010).  In  comparison  to 
XMARSHAL,  REDMAPS  provides  the  optimal  matching  solution. 
Therefore,  slightly  better  results  -  in  term  of  matching  rate  -  are 
expected.  However,  XMARSHAL  is  computationally  faster  than 
REDMAPS.  Since,  our  matching  rates  using  XMARSHAL  are  above 
the  clinically  acceptable  threshold,  we  prefer  XMARSHAL  due  to 
its  speed. 

In  our  results  on  clinical  data,  8%  of  reconstructions  have  a 
matching  rate  of  less  than  95%.  Although  Su  et  al.  suggested  that 
a  detection  rate  of  at  least  95%  is  required  for  an  accurate  dose  esti¬ 
mation  (Su  et  al.,  2005),  it  should  be  noted  that  seed  detection  and 
matching  rate  are  not  completely  equivalent.  As  mentioned  above, 
many  of  the  mismatches  occur  when  a  seed  projection  is  mistak¬ 
enly  matched  to  a  wrong  seed  projection,  which  is  closely  located 
to  the  correct  seed  projection.  In  this  case  the  reconstructed  seed  is 
considered  a  mismatch  but  the  localization  error  may  be  small. 
Therefore,  all  of  the  mismatched  seeds  cannot  be  categorized  as 
undetected  seeds.  As  explained,  by  using  more  than  three  images 
or  employing  a  matching  algorithm  with  optimal  outcome  such 
as  REDMAPS  (Lee  et  al.,  2011),  we  can  increase  the  matching  rate. 


We  implemented  our  algorithm  using  MATLAB  on  a  PC  with  an 
Intel  2.33  GHz  Core2  Quad  CPU  and  3.25  GB  of  RAM.  We  achieved 
an  average  time  of  19.4  s  per  data  set,  which  is  more  than  sufficient 
for  clinical  implementation.  The  first  level  of  the  optimization  is 
very  fast  since  only  10  seeds  per  image  are  used  in  the  reconstruc¬ 
tion  and  XMARSHAL  runs  on  an  0(N3)  runtime.  The  second  level  of 
the  algorithm  is  the  most  time  consuming  part,  since  a  full  match¬ 
ing  problem  must  be  solved  at  each  iteration.  However,  using  the 
outcomes  of  the  first-level  optimization  to  initialize  the  second  le¬ 
vel,  significantly  decreases  the  number  of  iterations  necessary  for 
convergence.  We  used  the  convergence  of  the  reconstruction  error 
as  the  stopping  condition  for  the  second  level,  which  is  a  conserva¬ 
tive  criterion.  As  an  alternative,  the  convergence  of  the  matching 
solution  can  be  used  -  similarly  to  the  first-level  optimization.  In 
this  case,  if  the  matching  solution  does  not  change  in  two  itera¬ 
tions,  the  optimization  will  terminate.  This  stopping  condition  is 
satisfied  in  significantly  fewer  iterations.  However,  it  was  observed 
during  the  simulations  -  although  rarely  -  that  a  matching  solution 
can  change  to  a  better  one  after  three  or  more  iterations.  Since,  the 
runtime  for  the  algorithms  is  already  acceptable,  we  decided  to 
choose  the  more  conservative  criterion,  in  order  to  increase  the 
seed  matching  rate.  As  mentioned  before,  the  third-level  optimiza¬ 
tion  and  seed  reconstruction  can  be  solved  using  (A.3)  and  (A.4) 
with  low  computational  cost. 

In  our  clinical  studies,  stranded  seeds  were  used.  We  used  the 
constant  center-to-center  distance  of  stranded  seeds  to  show  that 
no  significant  scaling  occurred  during  motion  compensation.  It 
should  be  noted  that  our  motion  compensation  and  reconstruction 
algorithms  do  not  rely  on  any  information  limited  to  stranded 
seeds  and  can  be  applied  for  loose  seeds  without  algorithmic 
modifications. 

In  this  work  the  seeds  were  manually  segmented  in  the 
images  by  one  observer  without  extensive  efforts  to  identify  all 
the  hidden  seeds.  Since  XMARSHAL  is  capable  of  maintaining 
above  97%  matching  rate  for  up  to  1mm  segmentation  error 
(Kon  et  al.,  2006),  our  motion  compensated  reconstruction  meth¬ 
od  can  successfully  reconstruct  the  seeds  as  long  as  the  manual 
seed  segmentation  error  is  below  this  threshold.  Considering  the 
small  dimensions  of  seed  projections,  this  level  of  accuracy  is 
easily  achievable.  However,  manual  seed  segmentation  is  a  te¬ 
dious  task  due  to  the  large  number  of  implanted  seeds.  Auto¬ 
matic  seed  segmentation  methods  are  available  that  have  a 
high  success  rate  in  identification  of  125I  (Tubic  et  al.,  2001b) 
or  103Pd  seeds  (Kuo  et  al.,  2010),  even  in  the  presence  of  over¬ 
lapping  seed  projections.  These  methods  can  be  used  to  segment 
the  majority  of  seeds  in  the  images.  The  missing  seeds  can  be 
identified  and  the  false  positives  can  be  removed  with  manual 
intervention.  Integration  of  an  automatic  segmentation  algorithm 
could  facilitate  the  use  of  intraoperative  dosimetry  in  a  clinical 
environment. 

We  assumed  that  the  C-arm  rotates  around  its  PA.  This  assump¬ 
tion  -  resulting  in  a  horizontal  motion  of  the  seed  projections  in 
the  images  -  was  used  in  the  first-level  optimization  to  justify 
the  selection  of  seeds  from  the  top  and  bottom  of  every  image  as 
candidates  for  matching  seeds.  If  the  C-arm  rotates  around  its  SA, 
the  seeds  move  vertically  in  the  images.  Therefore,  the  seeds  that 
appear  at  the  left  or  right  side  of  one  image  are  more  likely  to  ap¬ 
pear  at  the  left  or  right  side  of  other  images.  In  this  case,  these 
seeds  can  be  selected  as  candidates  for  matching  seeds  in  the 
first-level  optimization.  Similarly  to  our  case,  the  motion  along 
the  up-down  direction,  and  perpendicular  to  the  plane  of  rotation 
would  be  compensated. 

Since  we  did  not  use  radio-opaque  fiducials,  the  C-arm  could  be 
positioned  to  capture  the  seeds  close  to  the  center  of  the  detector. 
In  this  situation,  the  seed  segmentation  error  caused  by  geometric 
distortion  is  below  the  tolerance  level  of  XMARSHAL  (Kon  et  al., 
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2006).  Therefore,  we  did  not  correct  for  the  geometric  distortion.  If 
necessary,  the  dewarping  parameters  can  be  estimated  using  a  sin¬ 
gle  image  captured  at  the  center  of  the  rotation  span.  Jain  et  al. 
showed  that  for  small  rotation  spans,  the  dewarping  parameters 
obtained  from  a  center  image  can  effectively  correct  the  geometric 
distortions  of  all  the  images  (Jain  et  al.,  2007). 
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5.  Conclusions  and  future  work 

We  demonstrated  that  the  sole  measurement  of  rotation  angles 
of  a  C-arm  with  a  small  angle  span  in  a  single  plane,  combined  with 
a  motion  compensation  algorithm,  can  result  in  successful  prostate 
implant  reconstruction.  For  motion  compensation,  we  introduced  a 
three-level  optimization  method  to  compensate  for  C-arm  transla¬ 
tional  motions  in  the  Oj/wzw  plane  using  a  small  subset  of  seeds  as 
fiducials  to  gain  an  initial  estimate  of  the  C-arm  pose.  This  ap¬ 
proach  obviates  the  need  full  pose  tracking  with  external  trackers 
or  fiducials. 

In  a  clinical  study  of  100  data  sets  from  10  patients,  an  off- 
the-shelf  digital  protractor  or  C-arm  joint  encoders  were  used 
to  measure  the  rotation  angle  around  the  PA  of  a  C-arm,  while 
the  deviations  in  the  angle  around  the  left-right  axis  were  mea¬ 
sured  using  the  C-arm  joint  encoders.  Combined  seed  recon¬ 
struction  and  motion  compensation  led  to  on  average  seed 
matching  rate  of  98.5%,  projection  error  of  0.33  mm  and  19.8  s 
computational  time.  The  high  matching  rate,  insignificant  scaling 
effect,  low  projection  error  and  computation  time  show  the  fea¬ 
sibility  of  our  method  for  intraoperative  dosimetry  in  a  clinical 
setting. 

We  assumed  an  insignificant  motion  of  the  C-arm  along  xw 
axis.  This  assumption  was  validated  by  high  matching  rates 
and  small  projection  errors  in  our  clinical  study.  However,  if 
on  some  C-arms  this  motion  is  considerably  large,  a  3D  motion 
compensation  is  necessary.  This,  however,  may  suffer  from  the 
scaling  problem.  An  object  with  a  known  length  can  be  used 
to  estimate  the  scale.  Investigation  on  exploiting  the  length  of 
125I  seeds  or  seed  spacers  to  recover  the  scaling  factor  is  part 
of  our  future  work. 
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Appendix  A.  Equations  for  simultaneous  reconstruction  and 
motion  compensation 


Vv=(l-v9t/v).  (A.5) 

Eqs.  (A.3)  and  (A.4)  can  be  concatenated  into  a  matrix  form  as 
below. 

Ax  =  b,  (A.6) 

x=[s'1  s'N  5\  ■■■  4;'.  (A.7) 
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In  order  to  find  the  optimal  2D  offsets  of  M  sources  (M  images) 
and  at  the  same  time,  find  the  3D  position  of  N  implanted  seeds,  we 
solve  the  following  optimization  problem. 

(s;,SJ)  =  argminj  (A.l) 

N  M 

I  =  E  E(s<  -  %  -  Sj)\I  -  v9t/v)  (Si  -  Qj  -  Sj),  (A.2) 

i= 1  j= 1 


At  this  point,  the  set  of  linear  equations  in  (A.6)  is  under-deter¬ 
mined  and  the  matrix  A  in  (A.8)  is  singular.  However,  we  fixed 
the  first  source  in  space;  hence  S\  =  0.  In  addition,  we  assumed  that 
the  C-arms  move  only  in  Oywzw  plane.  This  places  another  con¬ 
straint  on  the  equations  in  the  form  of  Sjx  =  0,  je{  1,...,M}.  By 
removing  the  rows  and  columns  corresponding  to  these  known 
variables  from  A  and  removing  the  corresponding  entries  in  x  and 
b,  Eq.  (A.6)  can  be  solved. 


where  J  is  the  total  reconstruction  cost,  qj  is  the  initial  position  of  jth 
source  calculated  form  the  joint  angle  readings,  and  Sj  is  its  corre¬ 
sponding  offset.  The  minimality  necessary  conditions  imply  that: 
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Purpose:  Accurate  localization  of  prostate  implants  from  several  C-arm  images  is  necessary  for 
ultrasound-fluoroscopy  fusion  and  intraoperative  dosimetry.  The  authors  propose  a  computational 
motion  compensation  method  for  tomosynthesis-based  reconstruction  that  enables  3D  localization 
of  prostate  implants  from  C-arm  images  despite  C-arm  oscillation  and  sagging. 

Methods:  Five  C-arm  images  are  captured  by  rotating  the  C-arm  around  its  primary  axis,  while 
measuring  its  rotation  angle  using  a  protractor  or  the  C-arm  joint  encoder.  The  C-arm  images  are 
processed  to  obtain  binary  seed-only  images  from  which  a  volume  of  interest  is  reconstructed.  The 
motion  compensation  algorithm,  iteratively,  compensates  for  2D  translational  motion  of  the  C-arm 
by  maximizing  the  number  of  voxels  that  project  on  a  seed  projection  in  all  of  the  images.  This 
obviates  the  need  for  C-arm  full  pose  tracking  traditionally  implemented  using  radio-opaque  fidu- 
cials  or  external  trackers.  The  proposed  reconstruction  method  is  tested  in  simulations,  in  a  phan¬ 
tom  study  and  on  ten  patient  data  sets. 

Results:  In  a  phantom  implanted  with  136  dummy  seeds,  the  seed  detection  rate  was  100%  with  a 
localization  error  of  0.86  ±  0.44  mm  (Mean  ±  STD)  compared  to  CT.  For  patient  data  sets,  a  detec¬ 
tion  rate  of  99.5%  was  achieved  in  approximately  1  min  per  patient.  The  reconstruction  results  for 
patient  data  sets  were  compared  against  an  available  matching-based  reconstruction  method  and 
showed  relative  localization  difference  of  0.5  ±  0.4  mm. 

Conclusions:  The  motion  compensation  method  can  successfully  compensate  for  large  C-arm 
motion  without  using  radio-opaque  fiducial  or  external  trackers.  Considering  the  efficacy  of  the 
algorithm,  its  successful  reconstruction  rate  and  low  computational  burden,  the  algorithm  is  feasible 
for  clinical  use.  © 2011  American  Association  of  Physicists  in  Medicine.  [DOI:  10.1118/1.3633897] 

Key  words:  tomosynthesis,  brachy therapy,  seed  reconstruction,  motion  compensation,  C-arm 


I.  INTRODUCTION 

Since  its  advent  in  the  early  1980s,  ultrasound-guided  pros¬ 
tate  brachytherapy  (hereafter  brachytherapy)  has  become  a 
definitive  treatment  option  for  prostate  cancer — the  leading 
cancer  among  men  in  the  United  States  in  2010  (Ref.  1) — 
with  outcomes  comparable  to  the  radical  prostatectomy  that 
is  considered  as  the  gold  standard.2^  The  goal  of  brachy¬ 
therapy  is  to  kill  the  cancer  in  the  prostate  gland  with  radia¬ 
tion  by  permanently  implanted  radioactive  125I  or  103Pd 
capsules  (seeds).  Seed  positions  are  carefully  planned  to 
deliver  a  lethal  radioactive  dose  to  the  cancerous  prostate, 
while  maintaining  a  tolerable  dose  to  the  urethra  and  rectum. 
The  brachytherapist  delivers  the  seeds  using  needles  under 


visual  guidance  from  transrectal  ultrasound  (TRUS)  and 
qualitative  assessment  from  frequently  acquired  fluoroscopy 
images.5 

The  success  of  brachytherapy  depends  on  accurate  place¬ 
ment  of  the  seeds.  However,  prostate  motion  and  deforma¬ 
tion,6  needle  bending,  prostate  swelling,7  seed  migration,8 
and  human  and  system  calibration  errors  can  result  in  seed 
misplacement  which,  in  turn,  can  lead  to  underdosed  regions 
or  over-radiation  of  the  surrounding  healthy  tissue.  In  current 
brachytherapy  practice,  the  implant  is  quantitatively  assessed 
using  CT,  postoperatively.  In  case  of  major  underdosing, 
external  beam  radiation  is  applied  as  an  adjunct.  Intraopera¬ 
tive  dosimetry  can  provide  the  physicians  with  quantitative 


5290  Med.  Phys.  38  (1 0),  October  201 1 


0094-2405/201 1/38(1 0)/5290/1 3/$30.00 


©  201 1  Am.  Assoc.  Phys.  Med.  5290 


5291 


Dehghan  etal .:  Prostate  implant  reconstruction  using  motion-compensated  tomosynthesis 


5291 


dose  assessment  in  the  operating  room  and  enable  them  to 
adjust  the  position  and  number  of  the  remaining  seeds  to 
compensate  for  the  developing  cold  spots.9-11 

Three  dimensional  localization  of  the  implanted  seeds, 
registered  to  the  prostate  anatomy,  is  required  for  dose  calcu¬ 
lation.  TRUS  provides  sufficient  soft  tissue  contrast  to  delin¬ 
eate  the  prostate  boundaries.  However,  despite  significant 
efforts  in  seed  localization  from  ultrasound,12-18  robust  seed 
segmentation  in  ultrasound  is  not  yet  possible.  It  was  shown 
that  up  to  25%  of  the  seeds  can  be  missed  even  through  man¬ 
ual  segmentation  of  B-mode  images.12 

Mobile  C-arms  are  routinely  used  in  the  contemporary 
prostate  brachytherapy  for  implant  visualization.  However, 
the  prostate  cannot  be  visualized  in  the  C-arm  images. 
Therefore,  TRUS -fluoroscopy  fusion  offers  itself  as  a  practi- 
cal  solution  for  intraoperative  dosimetry.  ’  -  In  these 
methods,  the  seeds  reconstructed  from  C-arm  images  are 
spatially  registered  to  the  prostate  volume  visible  in  TRUS 
images.  The  delivered  dose  to  the  prostate  is  evaluated  and 
the  plan  is  modified,  accordingly. 

The  reconstruction  of  the  implanted  seeds  in  3D  space 
from  several  x-ray  images  has  been  widely  studied.23-38 
These  efforts  can  be  categorized  into  two  major  groups.  In 
the  first  group,  2D  coordinates  of  the  seed  projection  centers 
are  identified  in  the  images  and  a  matching  problem  is 
solved  to  identify  the  corresponding  projections  of  each  seed 
in  different  images.  ’  -  These  methods  should  be  pre¬ 
ceded  with  a  complicated  seed  segmentation  method  to  pre¬ 
cisely  localize  the  seed  projection  centroids.39^11  It  is 
difficult,  or  sometimes  impossible,  to  localize  the  centroid  of 
each  individual  seed  projection  in  an  image  due  to  presence 
of  hidden  and  overlapping  seed  projections  (see  Fig.  1). 
Therefore,  manual  intervention  is  usually  necessary  in  the 
seed  segmentation  phase.  However,  even  after  manual  inter¬ 
vention,  some  seed  projections  can  remain  hidden.  Although 
some  seed  matching  algorithms  can  address  the  hidden  seed 
problem,  ’  ’  the  performance  of  these  algorithms  usu¬ 
ally  degrades  with  increasing  number  of  hidden  seeds. 

The  second  group  of  seed  reconstruction  methods  consists 

rsA  OO  0^7 

of  tomosynthesis-based  algorithms.  ’  -  The  tomosynthesis- 
based  reconstruction  methods  have  two  advantages  over 
the  seed  matching  methods.  First,  the  matching  problem  in 
the  presence  of  hidden  or  overlapping  seed  projections  is 
inherently  solved  by  tomosynthesis.  Therefore,  these  methods 
do  not  need  a  seed  matching  algorithm.  Second,  the 
tomosynthesis-based  reconstruction  methods  require  a  much 
simpler  seed  segmentation  algorithm,  as  they  do  not  rely  on 
localization  of  seed  projection  centroids  in  every  image.  A  bi¬ 
nary  image  that  only  separates  the  seed  projections  from  the 
background — without  localization  of  their  centers — suffices 
for  a  tomosynthesis-based  seed  reconstruction. 

Tomosynthesis-based  seed  reconstruction  is  especially 
attractive  for  reconstruction  of  125I  seeds,  which  have  a 
larger  projection  compared  to  103Pd  seeds.  Due  to  their  rela¬ 
tively  larger  seed  projections,  overlapping  and  hidden  seed 
projections  are  more  abundant  in  the  C-arm  images  of  125I 
implants.  Therefore,  seed  segmentation  for  matching-based 
reconstruction  is  considerably  more  difficult  for  125I  seeds. 


Fig.  1 .  A  typical  C-arm  image  of  an  implant  showing  some  of  the  overlap¬ 
ping  and  hidden  seeds.  Localization  of  the  seed  projection  centroids  for 
hidden  or  overlapping  seeds  is  difficult  or  sometimes  impossible  for  seed 
segmentation  methods. 


This  makes  tomosynthesis,  the  preferred  method  for  125I 
seed  reconstruction.  However,  it  should  be  noted  that 
tomosynthesis-based  reconstructions  can  be  used  to  recon¬ 
struct  103Pd  seeds  without  any  restrictions. 

Pokhrel  et  a/.38  introduced  a  seed  reconstruction  method 
based  on  forward  projection  using  cone-beam  CT  (CBCT). 
Similarly  to  tomosynthesis-based  methods,  their  algorithm 
does  not  rely  on  identification  of  seed  projection  centroids. 
However,  their  method  is  computationally  more  extensive 
compared  to  tomosynthesis-based  algorithms.  Also,  impor¬ 
tantly,  CBCT  requires  a  high-end  digitally  encoded  C-arm 
that  is  typically  not  available  in  brachytherapy.  In  addition, 
CBCT  demands  many  C-arm  images,  exposing  the  patient 
and  OR  crew  to  more  toxic  radiation. 

C-arm  pose — the  relative  positions  of  images  in  3D 
space — must  be  known  prior  to  seed  reconstruction. 
Although,  external  electromagnetic  or  optical  trackers  can 
yield  the  C-arm  pose,42  they  are  not  practically  viable  due  to 
their  cost  and  added  complexity.  The  C-arm  pose  can  be  also 
computed  using  radio-opaque  fiducials.24,43^17  However, 
fiducials  require  segmentation,41  may  overlap  with  the  anat¬ 
omy  of  interest,  occupy  precious  real  estate  in  the  image, 
and  are  not  part  of  the  standard  operating  room. 

C-arm  images  are  generally  acquired  by  rotating  the 
C-arm  around  the  patient.  In  ideal  cases,  C-arm  rotation 
angles  can  yield  accurate  pose.  However,  in  real  cases, 
unmeasured  C-arm  translational  motions  caused  by  oscilla¬ 
tion  and  sagging  lead  to  errors  in  the  pose  computation  and 
in  turn,  failure  of  seed  reconstruction. 


Medical  Physics,  Vol.  38,  No.  10,  October  2011 


5292 


Dehghan  etal Prostate  implant  reconstruction  using  motion-compensated  tomosynthesis 


5292 


Researchers  have  developed  approaches  to  reconstruct 
the  seeds  and  use  them  to  improve  the  pose  computation, 
iteratively.25,48,49  In  these  methods,  the  seeds  are  recon¬ 
structed  in  3D  using  an  initial  estimate  of  the  pose.  Then,  a 
motion  compensation  method  uses  the  reconstructed  seeds  to 
compensate  for  the  errors  in  pose  computation.  In  Ref.  49, 
we  compensated  for  2D  translational  motion  of  the  C-arm 
using  the  reconstructed  seeds,  in  a  matching-based  recon¬ 
struction  scheme.  The  initial  pose  was  obtained  from  meas¬ 
urements  of  C-arm  rotation  angles  without  external  trackers 
or  fiducials. 

The  aforementioned  motion  compensation  methods, 
including  our  own  work,49  were  all  developed  for  matching - 
based  seed  reconstruction  and  hence,  cannot  be  applied  to  a 
tomosynthesis-based  reconstruction  method.  Lee  et  al?1 
were  the  first  to  use  a  motion  compensation  method  within  a 
tomosynthesis-based  reconstruction.  They  used  a  radio¬ 
opaque  tracking  fiducial  [called  FTRAC  Ref.  (46)]  to  ini¬ 
tially  estimate  the  pose  of  a  C-arm.  At  the  beginning,  three 
images  with  the  best  corresponding  pose  computation  qual¬ 
ity — based  on  the  residual  error  of  the  pose  recovery  using 
FTRAC — were  used  to  reconstruct  some  candidate  seeds. 
Then,  the  reconstructed  candidate  seeds  were  used  to 
improve  on  the  pose  and  calibration  parameters  for  the 
remaining  images  in  a  process  they  called  “autofocus.” 
Finally,  the  seeds  were  reconstructed  using  all  the  images.  If 
FTRAC  is  not  used,  the  quality  of  the  initial  pose  computa¬ 
tion  is  not  known.  Therefore,  the  three  images  with  the  best 
pose  cannot  be  selected  to  initialize  the  reconstruction.  In 
addition,  without  FTRAC,  a  tomosynthesis-based  seed 
reconstruction  may  fail  to  reconstruct  an  adequate  number  of 
candidate  seeds  for  pose  correction,  since  initial  pose  com¬ 
putation  may  not  be  sufficiently  accurate. 

In  this  paper,  we  introduce  a  new  computational  motion 
compensation  algorithm  for  tomosynthesis-based  seed 
reconstruction.  This  method  compensates  for  the  C-arm 
motion  by  maximizing  the  number  of  seed  voxels  in  a  vol¬ 
ume  of  interest.  In  contrast  to  the  previous  work,  this  method 
does  not  rely  on  reconstructed  seeds  to  compensate  for  C- 
arm  motion.  Therefore,  it  can  be  used  to  compensate  for 
large  motions  that  prohibit  initial  reconstruction  of  a  suffi¬ 
cient  number  of  seeds  for  seed-based  motion  compensation. 
The  proposed  motion  compensation  method  is  especially  tar¬ 
geted  for  tomosynthesis-based  reconstruction.  Therefore,  our 
method  inherits  the  advantages  of  a  tomosynthesis-based 
reconstruction — such  as  requiring  a  simple  segmentation  and 
inherently  solving  the  hidden  seeds  problem — which  make  it 
the  preferred  choice  for  reconstruction  of  125I  seeds.  How¬ 
ever,  this  method  is  not  limited  to  reconstruction  of  125I 
seeds  and  can  be  used  to  reconstruct  103Pd  seeds  as  well. 

Similarly  to  Ref.  49,  we  initialize  the  pose  by  sole  mea¬ 
surement  of  C-arm  rotation  angles.  On  the  one  hand,  this 
obviates  the  need  for  full  pose  tracking  using  radio-opaque 
fiducials  or  external  trackers.  But,  on  the  other  hand,  this  ini¬ 
tial  pose  estimation  can  fail  to  provide  us  with  an  adequate 
number  of  seeds  for  seed-based  pose  correction  through 
tomosynthesis.  As  we  will  show  in  Sec.  Ill,  maximizing  the 
number  of  seed  voxels  in  a  volume  of  interest,  without 


explicit  reconstruction  of  any  seeds,  surmounts  this  obstacle 
and  yields  accurate  C-arm  pose  computations  for  successful 
seed  reconstruction. 

In  Ref.  49,  we  demonstrated  that  by  making  realistic  and 
practical  assumptions  in  defining  the  imaging  protocol  in  ac¬ 
cordance  with  clinical  limitations,  a  2D  motion  compensa¬ 
tion  scheme  will  result  in  a  clinically  acceptable  seed 
reconstruction.  In  this  paper,  we  build  our  motion  compensa¬ 
tion  on  the  same  assumptions. 

We  assume  that: 

1 .  The  images  are  taken  by  rotating  the  C-arm  around  its  pri¬ 
mary  axis  PA  (yw  axis  in  Fig.  2)  in  a  limited  angle  span, 
while  the  angle  around  the  secondary  axis  (SA)  is  fixed 
(see  Fig.  2). 

2.  C-arm  rotation  angles  are  measured. 

3.  The  intrinsic  parameters  of  the  C-arm,  such  as  source  to 
image  distance,  source  to  center  of  rotation  distance, 
image  center  and  image  resolution  are  known  and  do  not 
change  during  the  C-arm  rotation. 

4.  Significant  C-arm  motions  are  translational  motions  in  the 
Oywzw  plane  and  motion  along  is  negligible. 

Single-axis  rotation  of  the  C-arm  around  its  PA  is  com¬ 
mon  practice  in  contemporary  brachy therapy.  Usually,  the 
PA  is  approximately  aligned  with  the  patient’s  craniocaudal 
axis.  C-arm  rotation  angles  can  be  measured  using  the  device 
joint  encoders  (if  available),  digital  protractors  or  accelerom¬ 
eters.50  Our  results  in  Sec.  Ill  C  suggest  that  an  accuracy 
of  ±  1°,  which  is  provided  by  C-arm  joint  encoders,  is  suffi¬ 
cient  for  successful  reconstructions.  The  intrinsic  parameters 
of  the  C-arm  can  be  measured  preoperatively.  Since  the  span 
of  C-arm  rotation  in  a  clinical  setting  is  generally  restricted 
to  ±  10°  due  to  space  limitations,  the  intrinsic  parameters  do 
not  significantly  change.  Jain  et  al.51  showed  that  recalibra¬ 
tion  is  unnecessary  since  small  changes  in  the  calibration 


Fig.  2.  Schematic  of  a  C-arm  rotating  around  its  PA  (rotation  ©).  Rotation 
(D  shows  rotation  of  the  C-arm  around  its  SA.  The  homogeneous  world  coor¬ 
dinate  system  Oxwywzw  is  centered  at  the  center  of  rotation.  The  homogene¬ 
ous  source  coordinate  system  Oxsyszs  is  centered  at  the  source  position 
corresponding  to  each  image. 
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parameters  do  not  have  a  significant  effect  on  the  relative 
position  of  the  reconstructed  seeds.  Note  that  we  are  inter¬ 
ested  in  the  relative  position  of  the  seeds,  since  the  recon¬ 
structed  seeds  will  be  registered  to  the  prostate  anatomy  for 
dosimetry.22 

The  C-arm  forms  a  cantilever  at  its  connection  to  the 
body  of  the  device.  The  intensifier  is  heavy  and  its  weight 
creates  a  significant  torque  around  the  connection  point  that 
can  lead  to  significant  sagging  along  the  z  axis.  In  addition, 
due  to  the  length  of  the  C-arm,  forces  along  the  y  and  z  axes 
can  create  significant  torques  around  this  joint  (connection  to 
the  body)  and  can  cause  oscillation  in  the  C-arm.  However, 
the  forces  along  the  x  axis  cannot  produce  significant  torque 
around  this  joint.  Therefore,  the  motion  caused  by  the  forces 
along  the  x  axis  can  only  result  in  translation  of  the  whole 
C-arm  and  its  body  along  the  x  axis.  Due  to  the  heavy  weight 
of  the  C-arm,  small  forces  along  this  axis  cannot  cause  sig¬ 
nificant  motion  when  the  C-arm  wheels  are  locked. 

Assume  that  images  /;,  i  G  { 1 , . . . ,  M}  were  acquired  from 
a  set  of  seeds  located  at  sj  =  [sjX,Sjy,Sjz]T ,  j  G  {1,...,V}, 
while  the  C-arm  source  positions  were  located  at 
qt  =  [qix,qiy,qiz}T ,i  G  {1,  ...,Af},  where  (.)r  denotes  the 
transpose  of  a  vector  or  a  matrix.  It  can  be  shown  that,  using 
the  same  set  of  images  /,  with  the  same  rotation  angles,  any 
C-arm  source  position  configuration  q\  =  [q'ixl  q'iy,  q'iz]T , 
i  G  {1,  that  satisfies  (#■  —  q\)  =  h(qt  —  q\)  will  result 

in  a  set  of  reconstructed  seeds  (s[)  in  which,  {s^  —  qi) 
=  k(si  —  qi).  In  other  words,  a  3D  translational  motion  com¬ 
pensation  can  result  in  a  reconstruction  with  an  arbitrary 
scale  (2).  If  a  fiducial  is  used,  a  known  length  on  the  fiducial 
can  be  used  to  recover  the  scale.48  In  order  to  avoid  the  scal¬ 
ing  problem  without  a  fiducial,  we  take  advantage  of  the 
confined  motion  of  the  C-arm  and  assume  that  the  C-arm 
motion  along  xw  is  negligible.  Therefore,  we  assume  that  our 
initial  C-arm  pose  estimations  are  accurate  along  the  xw  axis 
(qix)  and  we  add  the  constraint  (q'ix  —  q\x)  =  (qix  —  q\x)  to 
our  equations. 

Our  assumption  about  2D  motion  of  a  C-arm  is  an 
approximation  to  the  C-arm  motion  pattern.  However,  we 
show  in  Sec.  Ill,  that  this  approximation  is  sufficiently  accu¬ 
rate  for  successful  reconstruction  of  brachytherapy  implants 
and  results  in  a  negligible  error  in  the  estimation  of  the  scale 
(Table  II). 

In  Sec.  II,  the  methods  for  tomosynthesis-based  seed 
reconstruction  and  motion  compensation  are  outlined.  Sec¬ 
tion  III  shows  our  numerical  simulation,  phantom,  and  clini¬ 
cal  results.  We  discuss  our  results  in  Sec.  IV,  followed  by 
conclusions  and  future  work  in  Sec.  V. 

II.  MATERIAL  AND  METHODS 
II.A.  Image  processing  and  labeling 

We  use  seed-only  C-arm  images  to  reconstruct  the  seeds. 
A  seed-only  image  is  a  binary  image  in  which  each  pixel  has 
a  value  of  1,  if  it  belongs  to  a  seed  projection  and  zero  other¬ 
wise.  An  example  of  such  an  image  is  shown  in  the  left  side 
of  Fig.  3.  Note  that  in  contrast  to  seed  segmentation  for 
matching-based  reconstruction  methods,  we  do  not  require 


Fig.  3.  Left:  a  dilated  seed-only  image,  right:  labeled  seed-only  image. 

the  seed  projection  centroids  to  be  localized.  We  relied  on 
local  thresholding  and  morphological  filtering  to  produce  the 
seed-only  images  as  explained  in  Refs.  19  and  52.  There  are 
several  other  methods  that  can  be  used  to  produce  the  seed- 
only  images.18,34,36,39^1  Therefore,  we  do  not  discuss  image 
processing  in  more  details  in  this  paper.  It  should  be  noted 
that  a  false  positive  projection  does  not  result  in  a  false  posi¬ 
tive  seed,  unless  there  are  corresponding  false  positives  in  all 
the  other  images,  and  this  is  very  unlikely  to  occur.  How¬ 
ever,  a  missing  seed  projection  in  one  image,  results  in  a 
missing  seed  in  the  reconstructed  seed  cloud.  Therefore, 
manual  identification  of  missing  seed  projections  is  neces¬ 
sary;  however,  removal  of  false  positives  is  precautionary. 

In  order  to  increase  the  likelihood  of  seed  detection  and 
compensate  for  small  pose  computation  errors,  the  seed-only 
images  are  dilated  with  a  disk  structural  element  of  radius  r 
(2-3  pixels  in  this  work). 

The  dilated  seed-only  images  are  then  labeled  using  the 
connected  component  labeling  algorithm.53  These  labeled 
images  are  later  used  to  detect  and  remove  false  posi¬ 
tives.35,37  More  details  are  discussed  in  Sec.  II  D.  A  portion 
of  a  labeled  image  is  shown  in  Fig.  3.  Note  that  one  or  more 
seed  projections  can  be  associated  with  one  label. 


II. B.  Volume  of  interest  reconstruction 


Figure  2  shows  the  geometry  of  a  C-arm  rotated  around 
its  primary  axis  (PA).  Every  point  s  in  the  world  homogene¬ 
ous  coordinate  system — centered  at  Oxwywzw — can  be  pro¬ 
jected  on  a  point  p  on  the  ith  segmented  image  homogeneous 
coordinate  system  using  the  following  equation: 


f/Px 


0  c* 

-f/Py  Cy 

0  1 


sTl  v  - 


:  P iS, 


(1) 


where  is  the  transformation  matrix  from  the  world  ho¬ 
mogeneous  coordinate  system  to  the  source  homogeneous 
coordinate  system  centered  at  the  x-ray  source  location  that 
corresponds  to  the  zth  image,  /  is  the  source  to  image  dis¬ 
tance,  px  and  py  are  pixel  spacings  along  the  horizontal  and 
vertical  axes  of  the  image,  cx  and  cy  are  the  coordinates  of 
the  image  center  and  s  represents  the  coordinates  of  s  in  the 
world  homogeneous  coordinate  system.  P,  is  a  3x4 
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projection  matrix  from  the  world  homogeneous  coordinate 
system  to  the  image  i  homogeneous  coordinate  system.  The 
point  p  has  a  pixel  value  i/q(p)  equal  to  1,  if  p  is  inside  a  seed 
projection  and  equal  to  0,  otherwise. 

We  assume  a  volume  of  interest  (VOI)  in  the  3D  space. 
For  every  voxel  v  in  the  VOI  with  coordinates  y  in  the  world 
coordinate  system,  the  voxel  value  is  defined  as: 

M 

i=  1 

where  M  is  the  number  of  images.  A  voxel  is  assumed  to 
belong  to  a  seed  cluster  in  3D  space,  if 

^(v)  =  M.  (3) 

This  means  that  voxel  v  belongs  to  a  seed  cluster,  if  it 
projects  on  a  seed  projection  in  all  of  the  images.  We  define 
S  as  a  set  that  contains  all  the  seed  voxels. 

After  populating  the  VOI,  the  seed  clusters  are  labeled 
using  the  connected  component  labeling  algorithm.53  We 
use  these  labeled  clusters,  their  relation  with  the  labeled 
images,  and  their  centers  and  volumes  in  Sec.  II  D  to  find  the 
seed  centroids  and  remove  the  false  positives. 

Since  we  are  only  interested  in  voxels  with  a  value  of  M, 
we  can  significantly  increase  the  computational  speed  for 
VOI  reconstruction  using  the  following  procedure. 

At  the  beginning,  all  the  voxels  are  initialized  with  a 
voxel  value  equal  to  zero.  In  the  first  iteration,  all  the  voxels 
in  the  VOI  are  projected  on  the  first  image  using  Eq.  (1).  If  a 
voxel  projects  on  a  seed  projection  in  this  image,  its  voxel 
value  is  increased  by  1,  otherwise  its  voxel  value  is  kept 
unchanged  (see  Fig.  4).  The  voxels  that  have  a  value  of  zero 
after  this  iteration  do  not  have  the  opportunity  to  acquire  a 
value  of  M  after  projection  on  the  subsequent  M—  1  images. 
Therefore,  in  the  second  iteration,  only  the  voxels  with  a 
voxel  value  of  1  are  projected  on  the  second  image  and  their 
voxel  values  are  updated  in  the  same  manner.  Likewise,  in 
the  zth  iteration  (/  <  M),  only  the  voxels  with  a  value  of 
(/  —  1)  are  projected.  This  decreases  the  number  of  projected 


V7 


Fig.  4.  Projection  of  two  voxels  on  a  seed-only  image.  In  this  projection,  the 
voxel  value  of  Voxel  1  is  increased  by  one  since  Voxel  1  projects  on  a  seed 
projection  in  this  image.  The  voxel  value  of  Voxel  2  is  unchanged. 


voxels  significantly  after  each  iteration  and  increases  the 
computational  speed  that  is  very  important  for  clinical  seed 
reconstruction.  Since  every  voxel  is  projected  on  M  images 
only,  the  maximum  voxel  value  is  equal  to  M.  This  forward- 
projection  approach  removes  the  risk  of  cross-talk  between 
voxels  that  may  occur  in  back-projection. 

II. C.  Motion  compensation 

The  pose  computation  problem  is  equivalent  to  Ending 
the  transformation  matrix  in  Eq.  (1).  This  matrix  can  be 
defined  using  the  following  equation: 


"0" 

-SRWS  - 

0 

/ 

.  or 

I 

where  SRW  is  the  rotation  matrix  from  the  world  to  the  source 
coordinate  frame,  /  is  the  distance  from  the  source  to  the  cen¬ 
ter  of  rotation,  and  3  =  [3X  3y  3Z]T  is  the  translational  motion 
of  the  C-arm  caused  by  oscillation  and  sagging.  We  can  initi¬ 
alize  a  pose  estimation  by  measuring  the  C-arm  rotation 
angles  that  define  SRW ,  and  setting  the  unknown  C-arm  trans¬ 
lational  motion  (<5)  equal  to  zero.  The  error  caused  by  assum¬ 
ing  (5  =  0  can  result  in  significant  pose  computation  errors 
and  consequently  in  unsuccessful  reconstructions.  Therefore, 
we  should  compensate  for  translational  motions  and  improve 
on  our  pose  computation. 

As  mentioned,  Lee’s  autofocus  method37  is  not  applicable 
to  our  problem,  due  to  the  absence  of  the  FTRAC.  Therefore, 
we  propose  a  different  motion  compensation  schemes. 

We  observed  that  the  cardinality  of  S — the  total  number 
of  seed  voxels  in  the  VOI — is  maximized  when  the  pose  is 
accurately  known.  Figure  5  shows  the  cardinality  of  S  as  a 
function  of  C-arm  translational  pose  errors  in  the  up-down 
direction  (along  zw)  and  perpendicular  to  the  plane  of  rota¬ 
tion  (along  yw).  This  figure  shows  a  simulated  case,  in  which 
the  poses  of  4  images  are  accurately  known,  and  the  errors 


Fig.  5.  The  total  number  of  seed  voxels  in  a  VOI  as  a  function  of  pose  esti¬ 
mation  errors.  Errors  are  in  the  up-down  direction  (along  zw)  and  perpendic¬ 
ular  to  the  plane  of  rotation  (along  yw). 
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are  added  to  the  5th  pose.  Note  that  the  cardinality  of  S  is  an 
integer- valued  function. 

In  the  motion  compensation  algorithm,  we  assume  that 
the  position  of  the  C-arm  corresponding  to  the  first  image 
(henceforth  the  first  C-arm)  is  fixed  in  the  3D  space  and 
compensate  for  2D  motion  of  the  rest  of  the  C-arm  positions 
by  solving  the  following  problem: 

S*  =  argmax||S||, 

Oi 

s.t.  5xi  =  0,  (5) 

i  e  {2, 

where  ||.||  denotes  the  cardinality  of  a  set. 

Note  that  unlike  motion  compensation  methods  intro¬ 
duced  in  the  previous  work,25,37,48,49  this  method  does  not 
require  seeds  to  compensate  for  pose  errors. 

The  optimization  function  in  Eq.  (5)  is  integer- valued. 
Furthermore,  it  has  several  local  maximums,  as  can  be  seen 
in  Fig.  5.  In  order  to  remedy  these  problems,  we  exploit  the 
covariance  matrix  adaptation  evolution  strategy  (CMA- 
ES).54  This  is  a  stochastic  and  gradient- free  numerical  opti¬ 
mization  method  suitable  for  nonlinear  and  nonconvex 
problems. 

Although  we  benefit  from  a  fast  forward-projection 
method  explained  in  Sec.  II  B,  populating  the  VOI  in  every 
iteration  can  be  time  consuming  depending  on  its  size  and 
resolution  and  the  number  of  images.  In  order  to  reduce  the 
computational  time,  the  motion  compensation  is  performed 
on  a  smaller  VOI  with  a  lower  resolution.  In  addition,  only  a 
portion  of  each  image  is  used  to  populate  the  smaller  VOI. 

Rotation  of  the  C-arm  around  its  PA  results  in  horizontal 
motion  of  the  seeds  between  images  in  a  way  that  seed  pro¬ 
jections  located  at  the  top  or  bottom  of  one  image  appear  at 
the  top  or  bottom  of  the  other  images.  Therefore,  we  use  a 
narrow  band  from  the  top  of  each  seed-only  image  to  popu¬ 
late  the  smaller  VOI  during  the  motion  compensation.  The 
size  of  the  VOI  during  motion  compensation  is  adjusted 
according  to  the  width  of  the  band.  We  also  observed  that 
extra  dilation  of  the  band  images  increases  the  capture  range 
of  the  optimization  algorithm  and  also  decreases  the  number 
of  iterations  required  to  obtain  a  sufficient  motion  compensa¬ 
tion.  Figure  6  shows  a  sample  of  a  band  image  used  for 


Fig.  6.  A  band  image  used  for  motion  compensation. 


motion  compensation.  Note  that  this  image  is  more  dilated 
compared  to  the  image  in  Fig.  3. 

II. D.  Seed  detection  and  false  positive  removal 

After  motion  compensation,  the  VOI  is  populated  using 
the  improved  pose  computations.  The  seed  voxels  are  labeled 
into  different  clusters  using  the  so  called  connected  com¬ 
ponent  labeling  algorithm.53  Unfortunately,  tomosynthesis- 
based  seed  reconstruction  is  prone  to  producing  false  positive 
(FP)  seeds,  due  to  the  small  number  of  images  used  (see 
Fig.  7).  These  false  positive  seeds  should  be  removed  before 
dose  calculation  to  avoid  overestimating  the  radiation  dose. 

The  FPs  are  generated  when  M  cone-shaped  back  projec¬ 
tions  from  M  seed  projections  intersect  or  touch  in  a  voxel, 
which  is  not  a  true  seed  voxel.  Therefore,  the  FP  clusters 
usually  have  small  volumes.  However,  due  to  errors  in  the 
pose  computation  (even  after  motion  compensation)  and  cal¬ 
ibration  parameters,  the  clusters  in  the  VOI  have  a  wide 
range  of  volumes  as  shown  by  a  histogram  of  the  cluster  vol¬ 
umes  after  motion  compensation  for  a  real  patient  in  Fig.  8. 
Indeed,  some  of  true  seed  clusters  can  have  small  volumes 
comparable  to  volume  of  a  false  positive  cluster.  Thus,  an 
FP  removal  method  purely  based  on  the  volume  of  clus- 

O/I  oz:  on 

ters  ’  can  also  remove  some  of  the  true  seeds.  Fee  et  al. 
used  an  optimal  coverage  problem  approach  and  a  greedy 
search  to  remove  the  FP  seeds.  They  found  the  minimum 
subset  of  the  reconstructed  seeds  that  covers  all  the  seed  pro¬ 
jections  in  all  the  images.  Fooking  at  Fig.  7,  one  can  see  that 
if  we  remove  any  of  the  true  seeds  from  the  reconstructed 
seeds,  we  cannot  cover  all  the  seed  projections  in  the  seed- 
only  images.  Therefore,  the  three  true  seeds  are  the  only 
subset  of  the  reconstructed  seeds  that  cover  all  the  seed 
projections.  However,  it  should  be  noted  that  due  to  seed 
projection  overlap  in  the  images,  the  smallest  subset  of  the 
reconstructed  seeds  that  cover  all  the  projections  has  signifi¬ 
cantly  fewer  members  than  the  number  of  implanted  seeds. 
Therefore,  we  use  information  about  the  cluster  volumes  to 
add  a  necessary  number  of  seeds  to  the  covering  subset  to 
reconstruct  all  the  implanted  seeds. 


Fig.  7.  A  false  positive  seed  (white  circle)  and  three  true  seeds  (black 
circles).  If  any  of  the  true  seeds  are  removed,  one  cannot  cover  all  the  seed 
projections  in  the  images. 


Medical  Physics,  Vol.  38,  No.  10,  October  2011 


5296 


Dehghan  etal Prostate  implant  reconstruction  using  motion-compensated  tomosynthesis 


5296 


Volume  of  Seed  Clusters  (voxels) 


Fig.  8.  Histogram  of  the  seed  cluster  volumes  for  a  real  patient.  Due  to  the 
wide  range  of  cluster  volumes,  a  predefined  volume  threshold  cannot 
remove  the  FPs. 

We  take  the  following  steps  to  identify  and  remove  the 
false  positives: 

1.  Clusters  with  large  volumes  are  separated  into  multiple 
clusters.  These  clusters  are  generated  when  two  seeds  are 
located  very  close  to  each  other.  Although  the  seed  posi¬ 
tions  are  planned  to  be  at  least  5  mm  away  from  each 
other,  due  to  the  seed  misplacements,  two  seeds  can  be 
sufficiently  close  to  each  other  to  form  a  combined  clus¬ 
ter.  In  this  paper,  we  examine  all  clusters  that  have  a  vol¬ 
ume  greater  than  the  median  volume  with  a  6-neighbor 
connected  component  labeling  (the  initial  labeling  is  per¬ 
formed  using  26-neighbor  connection).  Usually,  clusters 
that  are  barely  touching  can  be  separated  into  their  con¬ 
stituent  clusters.  In  addition,  we  divide  the  clusters  that 
have  a  volume  larger  than  ccvm  (a  =  2  in  this  work),  where 
vm  is  the  median  volume. 

2.  The  centroids  of  the  seed  clusters  are  calculated  as  seed 
candidates  and  reprojected  on  the  labeled  seed-only 
images. 

3.  At  this  point,  we  generate  a  Nt  x  M  assignment  table, 
where  Nt  is  the  total  number  of  reconstructed  seed  cent¬ 
roids  ( N  is  the  number  of  implanted  seeds).  Entry  (/,/)  of 
the  assignment  table  shows  the  label  of  the  seed  projec¬ 
tion  in  image  j  where  the  zth  seed  centroid  is  projected. 

4.  If  two  or  more  seed  centroids  project  on  similar  seed  pro¬ 
jections  in  all  the  images  (have  identical  rows  in  the 
assignment  table),  the  one  with  the  largest  cluster  volume 
is  saved  and  the  rest  are  removed,  unless  the  seeds  are 
from  a  separated  large  cluster.  The  seeds  that  are  sepa¬ 
rated  from  a  large  cluster  in  step  1  are  marked  and  will 
not  be  removed. 

5.  The  seed  centroids  that  project  on  at  least  one  unique  seed 
projection  in  one  image  are  marked  as  unique  seed  cent¬ 
roids  and  are  preserved  regardless  of  their  cluster  volume. 
If  we  remove  any  of  these  seeds,  we  cannot  cover  all  the 
seed  projections. 


6.  Seed  centroids  that  have  a  cluster  volume  smaller  than  a 
threshold  (20%  of  the  median  volume  in  this  work)  and 
are  not  one  of  the  unique  seeds  are  removed. 

7.  The  list  of  unique  seeds  is  updated  after  removal  of  small 
clusters.  Assume  Nu  unique  seeds  are  available  at  this 
stage.  The  rest  of  the  seeds  share  all  of  their  seed  projec¬ 
tions  with  other  seed  centroids.  Due  to  the  seed  projection 
overlap  (see  Figs.  1  or  3),  the  number  of  unique  seeds  is 
less  than  the  number  of  implanted  seeds. 

8.  Finally,  we  add  N  —  Nu  seeds  with  maximum  cluster  vol¬ 
umes  from  the  remaining  seeds  to  the  unique  seeds  to 
reconstruct  N  seeds  in  total. 

II. E.  Numerical  simulations 

Four  seed  clouds  were  simulated  based  on  four  realistic 
implant  plans  with  100,  108,  110,  and  130  seeds.  Seeds  were 
simulated  as  capsules  with  diameter  of  1  mm  and  length  of 

4.5  mm,  approximately  equal  to  the  radio-opaque  size  of  125I 
seeds.  The  relative  positions  of  the  seed  centroids  were 
imported  from  the  plan.  We  assumed  that  the  long  axis  of  the 
capsules  is  parallel  to  the  yw  axis.  Seed-only  images  were  syn¬ 
thesized  by  rotating  the  C-arm  around  its  PA  by  angles  of 
0°,  ±  5°,  and  ±  10°,  while  the  SA  angle  was  kept  constantly 
at  180°.  Translational  errors  of  0-5  mm  along  the  yw  axis  with 
steps  of  1  mm  and  0-20  mm  along  the  zw  with  steps  of  2  mm 
were  independently  applied  to  one  of  the  C-arm  poses.  All  of 
the  five  images  were  used  to  reconstruct  the  seeds,  with  and 
without  motion  compensation.  Intrinsic  parameters  of  a  GE 
OEC  9800  mobile  C-arm  were  used  in  the  simulations. 

II. F.  Phantom  validation 

A  CIRS  Model-053  prostate  brachytherapy  training  phan¬ 
tom  (CIRS  Inc.,  Norfolk,  VA)  was  used  in  our  phantom 
study.  An  experienced  brachytherapist  implanted  136 
dummy  stranded  seeds  using  26  needles,  based  on  a  realistic 
implant  plan  prepared  by  a  board-certified  medical  physicist. 
A  motorized  GE  OEC  9800  mobile  C-arm  was  used  to  ac¬ 
quire  five  images  by  rotating  the  device  around  its  PA  in  a 
20°  rotation  span  in  approximately  5°  intervals.  Rotation 
angles  were  measured  using  a  digital  protractor  attached  to 
the  C-arm  source  casing.  In  order  to  establish  a  ground  truth, 
the  phantom  was  also  scanned  with  a  Picker  PQ5000  CT 
scanner.  We  segmented  the  seeds  in  the  CT  volume  by 
thresholding. 

II. G.  Patient  study 

Ten  patients  were  implanted  with  100-135  (average  112) 
stranded  125I  seeds  at  the  British  Columbia  Cancer  Agency 
(Vancouver,  BC,  Canada).  The  patients  had  a  prescribed  dose 
of  144.0  Gy  and  an  average  prostate  target  volume  (PTV)  of 

54.5  cc.  For  each  patient,  five  images  were  taken  using  a 
motorized  GE  OEC  9800.  This  device  has  a  heavy  intensifier 
that  causes  significant  sagging  and  necessitates  motion  com¬ 
pensation  for  seed  reconstruction.  The  C-arm  was  rotated 
around  its  PA,  which  was  aligned  with  the  craniocaudal  axis 
of  the  patient.  The  images  were  taken  at  angles  of 
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approximately  0°,  ±  5°,  and  ±10°.  Rotation  angles  were 
measured  using  a  digital  protractor  or  the  device  joint 
encoders.  The  digital  protractor  had  a  resolution  of  0.1°  and 
the  joint  encoders  had  a  resolution  of  1°.  The  rotation  angle 
around  SA  was  fixed  at  180°.  However,  deviations  of  1°  were 
observed  according  to  the  C-arm  joint  encoders.  These  devia¬ 
tions  were  taken  into  account  during  the  seed  reconstruction. 

The  C-arm  was  calibrated  preoperatively.  We  assumed 
that  the  C-arm  intrinsic  parameters  were  constant  for  all 
the  rotation  angles  and  patients.  The  seeds  were  reconstructed 
in  a  65  x  80  x  70  mm3  VOI  with  a  voxel  size  of  0.25 
x  0.5  x  0.5  mm3  and  image  dilation  radius  of  2  or  3  pixels. 
During  the  motion  compensation,  band  images  with  band 
width  of  150  pixels  and  dilation  radius  of  6  pixels  were  used 
for  all  the  patients.  A65xl0x70  mm3  VOI  with  a  voxel 
size  of  1  x  1  x  1  mm3  was  used  to  achieve  higher  speeds. 

III.  RESULTS 

III.A.  Numerical  simulations 

For  numerical  simulations,  the  localization  error  was 
measured  as  the  distance  between  the  reconstructed  and  syn¬ 
thesized  seeds  after  a  rigid  registration.  Figure  9  shows  the 
seed  detection  rate  and  localization  error  of  the  reconstructed 
seeds  versus  the  introduced  pose  error.  As  it  can  be  seen, 
motion-compensated  seed  reconstruction  is  able  to  maintain 
high  seed  detection  rates  and  low  localization  errors,  despite 
the  presence  of  translational  pose  errors,  while  the  recon¬ 
struction  without  motion  compensation  fails. 


III.B.  Phantom  study 

The  seeds  reconstructed  using  the  motion-compensated 
tomosynthesis  were  compared  with  the  seeds  segmented  in 
CT  after  a  rigid  registration.  Although  the  CT  and  fluoros¬ 
copy  images  were  taken  at  different  times,  we  assumed  that 
the  phantom  deformation  and  seed  displacements  were  neg¬ 
ligible.  We  achieved  a  100%  seed  detection  rate  with 
0.86  ±  0.44  mm  (Mean  ±  STD)  localization  difference 
between  CT  and  C-arm-based  reconstructions. 

III.C.  Clinical  results 

For  our  clinical  data  sets,  we  reprojected  the  recon¬ 
structed  seeds  on  the  C-arm  images  as  shown  in  Fig.  10.  As 
it  can  be  seen,  hidden  and  overlapping  seeds  were  success¬ 
fully  reconstructed.  The  images  were  meticulously  inspected 
for  missing  seeds.  The  seed  detection  rate  for  each  patient  is 
reported  in  Table  I. 

Since  the  real  positions  of  the  seeds  were  unknown,  we 
compared  our  results  with  the  results  of  an  available  motion- 
compensated  matching-based  seed  reconstruction  method49 
after  a  rigid  registration  and  reported  the  registration  error  in 
Table  I. 

We  achieved  an  average  seed  detection  rate  of  99.5%, 
which  is  a  clinically  excellent  result.  Su  et  al55  showed  that  in 
125I  prostate  implants  a  seed  detection  rate  of  above  95%  is 
sufficient  to  achieve  clinically  accurate  dose  calculations.  Our 
seed  detection  rates  are  above  this  threshold  for  all  the 
patients.  The  seed  detection  rate  without  motion  compensation 
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Fig.  9.  Simulation  results,  showing  the  average  seed  detection  rate  and  localization  error  for  variable  pose  errors.  The  average  of  seed  detection  rate  for  errors 
along  yw  and  zw  are  shown  in  (a)  and  (b),  respectively,  for  reconstructions  with  and  without  motion  compensation.  The  mean  and  STD  of  localization  error  for 
errors  along  yw  and  zw  are  shown  in  (c)  and  (d),  respectively,  for  reconstruction  with  motion  compensation. 
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Fig.  10.  Reconstructed  seed  centroids  projected  on  the  C-arm  image. 


was  on  average  below  50%.  This  shows  the  necessity  of 
motion  compensation,  when  only  C-arm  rotation  angles  are 
measured. 

We  used  five  images  for  eight  of  the  patients.  For  patients 
9  and  10,  seed  detection  using  four  images  was  more  suc¬ 
cessful.  This  was  due  to  inaccurate  rotation  angle  measure¬ 
ment  for  one  of  the  images,  most  likely  caused  by  inaccurate 
reading  of  the  encoder  or  protractor  while  the  C-arm  was  still 
oscillating.  Grzeda  and  Fichtinger50  used  accelerometers  to 
measure  the  C-arm  rotation  angles  with  high  accuracy.  In 
addition,  the  accelerometer  can  sense  the  C-arm  oscillation 
and  send  a  signal  to  the  operator  when  the  oscillation  is  suffi¬ 
ciently  decayed.  Therefore,  using  accelerometers  results  in 
more  accurate  rotation  angle  measurement  and  sharper 
images. 

In  the  case  of  the  stranded  125I  seeds  used  in  our  clinical 
study,  the  seeds  in  a  strand  are  kept  at  a  fixed  center-to-center 


-50- 


w 


Fig.  11.  Reconstructed  seed  centroids.  Seeds  on  the  same  strand  are  con¬ 
nected  to  each  other. 

distance  of  10  mm.  In  order  to  gain  more  confidence  in  the 
reconstruction  results  and  confirm  that  no  significant  scaling 
occurred,  we  calculated  the  center-to-center  distance  of  the 
reconstructed  seeds  in  the  different  strands.56  Figure  1 1  shows 
a  reconstruction,  in  which  seeds  are  grouped  based  on  their 
strand.  Table  II  shows  the  mean  and  STD  of  interseed  spacing 
for  all  the  patients.  The  interseed  spacing  has  an  overall  aver¬ 
age  of  10.3  mm,  demonstrating  an  insignificant  scaling  effect. 

IV.  DISCUSSION 

IV.A.  Large  cluster  separation 

In  a  brachytherapy  plan,  seeds  are  located  at  least  5  mm 
apart  from  each  other.  Due  to  a  seed  misplacement  or  migra¬ 
tion,  two  seeds  may  be  located  sufficiently  close  to  each 
other  to  create  a  combined  seed  cluster  in  the  VOI.  In  the 
case  of  stranded  seeds,  two  consecutive  seeds  cannot  move 
toward  each  other  to  create  a  combined  cluster.  Neverthe¬ 
less,  two  adjacent  seeds  that  are  not  on  the  same  strand  may 
be  located  sufficiently  close  to  each  other  to  create  a  com¬ 
bined  cluster. 


Table  I.  The  clinical  results.  The  reconstruction  rate  is  assessed  visually  based  on  the  projection  of  the  reconstructed  seeds  on  the  images.  The  difference 
reports  the  registration  error  between  seed  locations  computed  using  the  proposed  method  and  an  available  seed  reconstruction  method. 


Patient  # 

Number  of  seeds 

Detection  rate  (%) 

Difference  (mm)  mean  ±  STD 

Dilation  radius  (pixel) 

1 

105 

100.0 

0.4  ±  0.3 

2 

2 

105 

100.0 

0.3  ±  0.4 

2 

3 

135 

100.0 

0.4  ±  0.3 

3 

4 

102 

99.0 

0.4  ±  0.3 

2 

5 

122 

100.0 

0.6  ±  0.4 

2 

6 

113 

100.0 

0.5  ±  0.3 

2 

7 

100 

98.0 

0.5  ±0.5 

2 

8 

120 

99.2 

0.9  ±  0.5 

3 

9a 

104 

98.1 

0.5  ±  0.3 

2 

10a 

115 

99.2 

0.6  ±  0.4 

2 

Tor  patients  9  and  10,  only  four  images  were  used. 
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Table  II.  The  mean  and  STD  of  the  distance  between  two  consecutive  seeds 
on  a  strand. 


Patient  # 

Seed  spacing  (mm)  mean  ±  STD 

1 

10.3  ±  0.4 

2 

10.3  ±0.3 

3 

10.3  ±0.3 

4 

10.3  ±0.3 

5 

10.3  ±0.5 

6 

10.2  ±  0.4 

7 

10.4  ±  0.6 

8 

10.0  ±0.5 

9 

10.4  ±0.3 

10 

10.2  ±  0.4 

Overall 

10.3  ±0.4 

Due  to  C-arm  calibration  and  pose  computation  errors 
(even  after  motion  compensation),  the  seed  clusters  have  a 
wide  range  of  volumes  (see  Fig.  8).  In  addition,  if  two  seeds 
are  very  close  to  each  other,  the  volume  of  the  merged  clus¬ 
ter  will  not  be  significantly  larger  than  a  single- seed  cluster. 
Therefore,  detection  of  multiple-seed  clusters  is  not  possible 
by  using  a  uniform  threshold  on  the  volume. 

As  mentioned,  125I  seeds  have  larger  seed  projections 
compared  to  103Pd  seeds,  which  lead  to  more  overlapping 
seed  projections  in  the  images,  which  in  turn  increase  the 
likelihood  of  having  combined  clusters  in  the  VOI.  In  addi¬ 
tion,  the  seed  density  can  affect  the  likelihood  of  formation 
of  combined  clusters.  Our  patients  had  a  seed  density  of 
approximately  2  seeds  per  milliliter  (total  number  of  seeds 
divided  by  PTV),  with  more  concentration  at  the  posterior- 
peripheral  region.3  In  treatment  plans  with  a  lower  seed  den¬ 
sity,  the  seeds  are  more  separated  and  merged  clusters  are 
less  likely  to  form. 

IV.B.  Determination  of  seed  dilation  radius 

Even  after  motion  compensation,  the  reconstruction  may 
suffer  from  minor  errors  in  the  rotation  angle  measurement, 
calibration  parameters,  and  geometric  distortion  as  well  as 
from  motion  along  the  axis.  Since  seed  clusters  are 
formed  at  the  intersection  of  rays  that  emanate  from  a  seed 
projection  toward  the  x-ray  source,  seed-only  image  dilation 
can  decrease  the  effects  of  the  aforementioned  errors  as  it 
can  increase  the  likelihood  of  seed  detection  by  increasing 
the  size  of  the  seed  projections.  However,  if  the  dilation  ra¬ 
dius  is  too  large,  the  seed  clusters  will  grow  in  size  and  ulti¬ 
mately  merge.  Therefore,  the  best  dilation  radius  should  be 
chosen  specifically  based  on  the  pose  and  parameter  estima¬ 
tion  errors.  We  used  a  dilation  radius  of  2  pixels  in  the  nu¬ 
merical  simulations  and  phantom  study  and  a  radius  of  2  or 
3  pixels  for  the  patient  data  sets  (see  Table  I).  However,  it 
should  be  noted  that  a  fixed  dilation  radius  of  6  pixels  was 
used  during  the  motion  compensation  phase  in  simulation, 
phantom,  and  clinical  studies.  Since  motion  compensation  is 
the  most  time  consuming  part  of  the  seed  reconstruction 
algorithm,  it  is  possible  to  use  a  fixed  dilation  radius  for 
motion  compensation,  then  adjust  the  dilation  radius  during 


final  VOI  reconstruction  and  seed  detection.  The  final  VOI 
reconstruction  and  seed  detection  take  approximately  5  s  of 
runtime. 

A  variable  dilation  radius  can  be  helpful  in  increasing  the 
detection  rate  without  increasing  the  large  clusters.  In  such  a 
method,  the  dilation  radius  will  be  larger  for  images  or  part 
of  images  that  are  affected  more  by  the  aforementioned 
errors,  while  a  small  dilation  radius  can  be  applied  where  the 
errors  are  small.  Investigation  on  variable  dilation  radius  is 
part  of  the  future  work. 

IV.C.  Localization  error 

In  contemporary  brachy therapy,  implants  are  assessed 
using  CT,  one  or  several  days  after  the  procedure.  C-arm 
images  are,  however,  taken  during  or  at  the  end  of  the  proce¬ 
dure,  while  the  patient  is  still  in  treatment  position.  In  addi¬ 
tion,  in  our  case,  the  TRUS  probe  was  still  partially  inside 
the  rectum  during  C-arm  imaging,  while  the  CT  scan  was 
performed  without  the  TRUS.  Due  to  prostate  swelling  dur¬ 
ing  and  after  brachytherapy,7  postimplant  seed  migration,8 
and  probe  pressure,  seed  positions  during  CT  scan  were  dif¬ 
ferent  from  the  position  of  the  seeds  when  the  C-arm  images 
were  taken.  Therefore,  CT  images  of  the  patient  could  not 
be  used  to  establish  a  confident  ground  truth  for  the  position 
of  the  seeds  in  3D.  For  this  reason,  we  relied  on  the  pro¬ 
jection  of  the  reconstructed  seeds  on  the  images  and  on  the 
comparison  with  the  results  of  another  reconstruction 
method  to  assess  our  reconstructions. 

It  was  shown  that  a  localization  uncertainty  of  less  than  2 
mm  results  in  less  than  5%  deviation  in  the  prostate  D90  (the 
minimum  dose  delivered  to  90%  of  the  prostate).57,58 
Although  we  could  not  measure  the  seed  localization  error 
for  our  clinical  data  sets,  the  localization  errors  in  our  nu¬ 
merical  simulations  and  phantom  studies  were  significantly 
lower  than  this  threshold. 

IV.D.  Computation  time 

We  implemented  our  algorithm  using  MATLAB  on  a  PC 
with  an  Intel  2.33  GHz  Core2  Quad  CPU  and  3.25  GB  of 
RAM.  MATLAB  implementation  of  CMA-ES  algorithm 
was  provided  by  N.  Hansen.59  The  CMA-ES  algorithm 
shows  faster  convergence  if  the  parameter  search  region  is 
limited.  Thus,  we  limited  the  search  region  to  ±  30  mm 
along  the  zw  axis  and  ±  3  mm  along  yw.  We  used  the  center 
of  mass  of  the  seed-only  images  to  initialize  the  displace¬ 
ment  along  yw.  Therefore,  displacements  of  larger  than  3 
mm  in  this  direction  could  be  recovered  in  the  simulation 
studies.  This  search  region  was  sufficiently  large  for  all  clini¬ 
cal  data  sets. 

The  criterion  to  terminate  the  optimization  was  set  to 
2000  function  evaluations.  This  resulted  in  a  constant  recon¬ 
struction  time  of  approximately  1  min  per  patient  (excluding 
the  production  of  seed-only  images).  Our  code  was  not  opti¬ 
mized  for  computational  speed.  We  expect  to  gain  faster  per¬ 
formance  using  an  optimized  C  +  +  implementation.  Band 
images  and  a  smaller  VOI  with  lower  resolution  were  used 
during  the  motion  compensation  phase  to  decrease  the 
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computation  time.  Investigation  on  the  optimal  image  band 
width  and  the  size  and  resolution  of  the  VOI  for  the  least 
computational  cost  are  part  of  our  future  work. 

On  our  patient  data  sets,  we  achieved  an  average  seed 
detection  rate  of  99.5 %  with  computational  time  of  approxi¬ 
mately  1  min  per  patient.  Similar  detection  rates  were 
reported  using  previously  published  tomosynthesis-based 
reconstmction  methods.  In  particular,  Lee  et  al 31  reported  an 
average  detection  rate  of  98.8%  in  approximately  100  s  per 
patient  and  Bmnet-Benkhoucha  et  al.36  reported  an  average 
detection  rate  of  96.7%  with  36.5  s  average  computational 
time.  Brunet-Benkhoucha  et  al.  used  a  radiotherapy  simula¬ 
tor,  which  is  a  precisely  calibrated  and  accurately  tracked  de¬ 
vice.  Hence,  they  did  not  require  motion  compensation.36  As 
discussed  before,  Lee  et  al31  used  the  FTRAC  (Ref.  46)  to 
initialize  a  pose  estimation  and  also  choose  the  best  images  to 
reconstmct  some  seeds  for  seed-based  motion  compensation. 
The  same  radio-opaque  fiducial  was  also  used  by  Jain  et  al.30 
Kon  et  al.31  and  Lee  et  al 32  to  reconstruct  the  seeds  using  a 
matching-based  approach.  In  these  works,  the  pose  computa¬ 
tion  accuracy  provided  by  the  FTRAC  was  sufficient  for  high 
detection  rates  without  motion  compensation.  As  mentioned, 
employing  such  a  fiducial  requires  an  additional  segmentation 
task.  Furthermore,  image  acquisition  in  presence  of  this  fidu¬ 
cial  is  more  complicated  in  order  to  avoid  an  overlap  between 
the  fiducial  image  and  the  seed  projections. 

In  our  previous  work,49  we  achieved  an  average  seed 
reconstruction  rate  of  98.5%  with  average  computational 
time  of  19.8  s  per  patient  using  three  images  in  a  motion- 
compensated  matching-based  reconstruction.  Although  the 
detection  rate  in  the  current  paper  is  only  slightly  better  than 
our  previous  work,  the  true  advantage  of  the  current  work  is 
in  enabling  motion  compensation  with  tomosynthesis-based 
reconstruction.  As  discussed  earlier,  matching-based  seed 
reconstruction  methods  require  a  more  complicated  seed 
segmentation  algorithm  as  they  require  the  seed  projection 
centroids,  which  are  difficult  to  localize  in  the  presence  of 
hidden  and  overlapping  projections.  Especially  for  125I  seeds 
that  have  relatively  longer  seed  projections,  overlapping 
seeds  are  more  common  in  the  projection  images.  For  this 
reason,  in  our  previous  work,  we  relied  on  manual  seed  seg¬ 
mentation  which  is  a  time  consuming  task.  Compared  to 
matching-based  reconstructions,  tomosynthesis-based  recon¬ 
struction  methods  require  a  simple  image  processing  step  to 
separate  the  seed  projection  regions  from  the  background. 
Therefore,  our  current  work  introduces  an  alternative  solu¬ 
tion  for  seed  reconstruction,  which  is  especially  attractive 
for  reconstruction  of  125I  seeds.  We  should  emphasize  that 
the  motion  compensation  method  proposed  in  our  previous 
work  is  not  applicable  to  tomosynthesis. 

As  mentioned  in  Sec.  II  C,  rotation  of  the  C-arm  around 
its  PA  results  in  horizontal  movement  of  the  seed  projections 
between  images.  If  the  C-arm  rotates  around  its  SA,  the  seed 
projections  move  along  vertical  lines  in  the  images.  There¬ 
fore,  vertical  bands  from  the  sides  of  the  images  can  be  used 
for  motion  compensation.  Similarly  to  our  case,  the  motion 
in  the  up-down  direction  and  perpendicular  to  the  plane  of 
rotation  should  be  compensated. 


We  used  the  constant  center-to-center  distance  of  the 
stranded  seeds  to  show  that  no  significant  scaling  occurred. 
However,  the  motion  compensation  and  reconstruction 
methods  do  not  rely  on  any  information  limited  to  stranded 
seeds.  Therefore,  we  expect  similar  performance  for  non- 
stranded  seeds. 

The  motion  compensation  method  can  be  extended  to  use 
more  images  at  the  expense  of  computational  time.  Increas¬ 
ing  the  number  of  images  can  reduce  the  number  of  false 
positives  and  increase  the  seed  detection  rate.  It  can  also 
decrease  the  likelihood  of  formation  of  merged  clusters  by 
decreasing  the  volume  of  seed  clusters.  The  same  effect  can 
be  achieved  by  using  a  wider  rotation  span. 

In  our  case,  we  assumed  that  the  image  geometric  distor¬ 
tion  was  negligible  as  we  positioned  the  C-arm  to  capture 
the  seed  projections  close  to  the  center  of  the  image.  How¬ 
ever,  correction  of  the  geometric  image  distortion  can 
increase  the  seed  detection  rate  and  improve  the  localization 
accuracy.  Although  image  distortion  is  pose  dependent,  Jain 
et  al.51  showed  that  in  small  rotation  spans,  the  correction 
parameters  obtained  from  an  image  acquired  in  the  center  of 
the  rotation  span  can  considerably  correct  for  the  distortion 
of  all  the  images  with  insignificant  relative  deviation  in 
reconstructed  seed  positions. 

V.  CONCLUSIONS  AND  FUTURE  WORK 

We  introduced  a  computational  2D  motion  compensation 
algorithm  for  tomosynthesis-based  seed  reconstruction  from 
C-arm  images.  We  initialized  the  C-arm  pose  using  sole 
measurements  of  rotation  angles,  and  we  compensated  for 
C-arm  motions  in  the  up-down  direction  and  perpendicular 
to  the  plane  of  rotation  by  maximizing  the  number  of  seed 
voxels  in  the  volume  of  interest.  Our  method  does  not 
require  reconstructed  seeds  for  motion  compensation.  There¬ 
fore,  it  can  be  used  to  recover  from  severe  pose  errors  that 
inhibit  reconstruction  of  a  sufficient  number  of  initial  seeds. 

In  a  clinical  study  on  ten  patients,  we  measured  the  C-arm 
rotation  angle  around  its  PA  using  a  digital  protractor  and 
C-arm  joint  encoders.  Seed  reconstruction  with  motion  com¬ 
pensation  achieved  an  average  seed  detection  rate  of  99.5%, 
which  is  a  clinically  excellent  result,  with  a  1  min  per  patient 
reconstruction  time.  We  also  achieved  100%  seed  detection 
rate  with  0.86  ±  0.44  mm  localization  error  in  a  phantom 
study. 

Our  motion  compensation  algorithm  obviates  the  need  for 
full  pose  tracking  with  radio-opaque  fiducials  or  other  exter¬ 
nal  trackers.  Considering  the  simplicity  of  implementation 
and  high  seed  detection  rates,  our  algorithm  appears  to  be 
feasible  for  clinical  application. 

Two  dimensional  motion  of  the  C-arm  was  an  essential 
assumption  in  this  work.  Extension  of  our  motion  compensa¬ 
tion  to  3D  and  removing  the  scaling  effect  using  the  inter¬ 
seed  spacing  are  part  of  the  future  work. 
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In  this  paper,  vibro-elastography  (VE),  an  ultrasound-based  method  that  creates  images  of  tissue  visco¬ 
elasticity  contrast,  is  evaluated  as  an  imaging  modality  to  visualize  and  segment  the  prostate.  We  report 
a  clinical  study  to  characterize  the  visibility  of  the  prostate  in  VE  images  and  the  ability  to  detect  the 
boundary  of  the  gland.  Measures  for  contrast,  edge  strength  characterized  by  gradient  and  statistical 
intensity  change  at  the  edge,  and  the  continuity  of  the  edges  are  proposed  and  computed  for  VE  and 
B-mode  ultrasound  images.  Furthermore,  using  MRI  as  the  gold  standard,  we  compare  the  error  in  the 
computation  of  the  volume  of  the  gland  from  VE  and  B-mode  images.  The  results  demonstrate  that  VE 
images  are  superior  to  B-mode  images  in  terms  of  contrast,  with  an  approximately  six  fold  improvement 
in  contrast-to-noise  ratio,  and  in  terms  of  edge  strength,  with  an  approximately  two  fold  improvement  in 
the  gradient  in  the  direction  normal  to  the  edge.  The  computed  volumes  show  that  the  VE  images  provide 
an  accurate  3D  visualization  of  the  prostate  with  volume  errors  that  are  slightly  lower  than  errors  com¬ 
puted  based  on  B-mode  images.  The  total  gland  volume  error  is  8.8  ±  2.5%  for  VE  vs.  MRI  and  10.3  ±  4.6% 
for  B-mode  vs.  MRI,  and  the  total  gland  volume  difference  is  -4.6  ±11.1%  for  VE  vs.  MRI  and  -4.1  ±17.1% 
for  B-mode  vs.  MRI,  averaged  over  nine  patients  and  three  observers.  Our  results  show  that  viscoelastic 
mapping  of  the  prostate  region  using  VE  images  can  play  an  important  role  in  improving  the  anatomic 
visualization  of  the  prostate  and  has  the  potential  of  becoming  an  integral  component  of  interventional 
procedures  such  as  brachytherapy. 

©  201 1  Elsevier  B.V.  All  rights  reserved. 


1.  Introduction 

Prostate  cancer  is  the  most  prevalent  type  of  cancer  among  men 
and  is  projected  to  affect  24,600  men  in  Canada  (Canadian  Cancer 
Society,  2010)  and  217,730  in  the  United  States  (National  Cancer 
Institute,  2010)  in  2010.  It  is  the  most  numerous  cancer  diagnosed 
in  European  men  (382,000  cases  in  2008)  (Ferlay  et  al.,  2010).  Com¬ 
mon  treatment  options  include  brachytherapy  and  radical  prosta¬ 
tectomy.  Low  dose  rate  (LDR)  prostate  brachytherapy  is  generally 
used  for  early  stage,  intra-capsular  prostate  cancer  and  has  rapidly 
gained  acceptance  due  to  its  highly  successful  clinical  results  (Mor¬ 
ris  et  al.,  2009).  In  this  treatment,  40-150  small  radioactive  seeds 
(Iodine-125  or  Palladium-103)  are  inserted  through  the  perineum 
and  permanently  implanted  into  the  prostate  and  periprostatic  tis¬ 
sue.  In  high  dose  rate  (HDR)  brachytherapy,  temporary  catheters 
are  placed  inside  the  prostate  which  allow  the  placement  of  high 
dose  rate  sources  delivering  the  radiation  treatment  over  a  number 
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of  fractions,  typically  over  a  few  days.  Radical  prostatectomy  is  a 
surgical  option  in  which  the  prostate  is  removed  either  by  laparo¬ 
scopic  or  open  surgery.  The  surgical  margin  within  which  the  pros¬ 
tate  is  removed  depends  on  the  stage  of  the  disease. 

Effective  treatment  of  prostate  cancer,  regardless  of  the  treat¬ 
ment  method  used,  requires  accurate  visualization  of  the  gland 
and  its  surrounding  region.  Accurate  delineation  of  the  prostate 
and  appropriate  visualization  of  the  prostatic  region  has  the  poten¬ 
tial  to  reduce  some  of  the  possible  side-effects  of  the  current  treat¬ 
ment  methods.  These  complications  include  urinary  incontinence, 
impotence,  and  damage  to  the  rectum  and  urethra  (Thompson 
et  al.,  2007). 

Ultrasound  is  the  most  commonly  used  modality  for  imaging  of 
the  prostate.  This  is  due  to  its  availability,  safety  and  ease  of  use. 
However,  ultrasound  B-mode  images  do  not  always  delineate  the 
prostate  reliably.  As  a  result,  prostate  boundary  extraction  becomes 
a  highly  subjective  process  (Smith  et  al.,  2007).  This  is  observed 
specifically  at  the  base  of  the  gland,  where  the  prostate  merges  with 
the  bladder  neck,  and  the  apex,  where  it  blends  into  the  pelvic  floor 
muscles.  It  has  been  shown  that  user  segmentation  variability  is 
large  in  these  areas  (Choi  et  al.,  2009;  Tong  et  al.,  1998). 

Many  attempts  have  been  made  to  improve  the  visibility  of  the 
prostate  in  ultrasound  B-mode  images.  These  vary  from  the 
processing  of  the  images  (Sahba  et  al.,  2005;  Pathak  et  al.,  2000), 
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to  the  use  of  additional  information  from  other  modalities  such  as 
MR  (Daanen  et  al.,  2006).  Another  recently  developed  option  is  the 
use  of  ultrasound  elastography. 

Elastography  (Ophir  et  al.,  1991, 1996)  is  a  promising  technique 
for  imaging  soft  tissues  and  relies  upon  measuring  tissue  strain  in 
response  to  a  mechanical  excitation.  Indeed,  when  compressed  by 
an  external  mechanical  exciter,  e.g.,  by  the  inward  motion  of  the 
ultrasound  transducer  in  the  image  axial  direction,  softer  tissue 
will  compress  more  than  stiffer  tissue,  and  therefore  experience 
larger  strain,  which  can  be  measured  and  displayed  by  processing 
the  ultrasound  echo  data.  Alternatively,  tissue  vibration  induced  by 
the  exciter  can  be  measured  with  Doppler  ultrasound  (Lerner  et  al., 
1988),  with  larger  vibrations  corresponding  to  softer  tissue.  In 
transient  elastography,  the  propagation  of  a  shear  wave  is  imaged 
with  parallel  receive  ultrasound  (Tanter  et  al.,  2002).  When  the 
excitation  is  dynamic,  the  shear  modulus  can  be  estimated  as  a 
complex  function  of  frequency,  thus  providing  information  on 
the  viscoelastic  properties  of  tissue.  In  the  last  few  years  various 
clinical  applications  of  elastography  have  been  reported  in  the  lit¬ 
erature.  These  include,  but  are  not  limited  to,  breast  lesions  (Garra 
et  al.,  1997;  Sinkus  et  al.,  2000;  Kadour  and  Noble,  2009;  Li  et  al., 
2009),  liver  fibrosis  (Castera  et  al.,  2005;  Huwart  et  al.,  2008;  Yin 
et  al.,  2007),  vascular  vulnerable  plaque  (Schaar  et  al.,  2003),  elastic 
properties  of  skeletal  muscle  (Dresner  et  al.,  2001),  thyroid  gland 
tumors  (Lyshchik  et  al.,  2005),  and  assessment  of  thermal  tissue 
ablation  (Wu  et  al.,  2001)  and  for  the  detection  of  prostate  cancer 
(Cochlin  et  al.,  2002;  Souchon  et  al.,  2003;  Miyagawa  et  al.,  2009; 
Zhang  et  al.,  2008;  Pallwein  et  al.,  2007;  Gravas  et  al.,  2009;  Kamoi 
et  al.,  2008;  Fleming  et  al.,  2009). 

Elastography  has  been  shown  to  be  promising  in  improving  the 
visibility  of  the  prostate  gland  and  the  cancer  within  it.  In  Egorov 
et  al.  (2006)  the  stress  pattern  on  the  rectal  wall  is  directly  mea¬ 
sured  with  the  use  of  a  transrectal  probe  equipped  with  a  pressure 
sensor  array.  Temporal  and  spatial  changes  in  the  stress  pattern 
provide  information  for  calculating  prostate  features  such  as  size, 
shape  and  hardness.  Phantom  and  in  vivo  results  show  that  such 
a  method  has  the  potential  to  replace  digital  rectal  examination 
(DRE).  The  usefulness  of  elastography  was  evaluated  on  311  pa¬ 
tients  in  Miyagawa  et  al.  (2009).  They  showed  that  the  sensitivity 
of  elastography  and  elastography  +  TRUS  imaging  in  detecting  can¬ 
cer  (confirmed  by  biopsy)  is  higher  than  that  of  DRE  or  TRUS  only. 
A  higher  prostate-specific  antigen  (PSA)  level  and  smaller  prostate 


volume  are  reported  to  increase  the  sensitivity  of  elastography  and 
elastography  +  TRUS.  However,  the  high  frequency  of  false-positive 
elastography  results  and  difficulty  in  the  detection  of  cancer  in  the 
peripheral  zone  are  two  main  problems  reported  in  their  work. 

Vibro-elastography  is  a  dynamic  ultrasound  elastography  meth¬ 
od  (Turgay  et  al.,  2006)  which  models  viscoelastic  properties  of  tis¬ 
sue.  The  approach  is  illustrated  in  Fig.  1.  The  technique  relies  on 
the  continuous  real  time  acquisition  of  unprocessed  ultrasound 
echo  data  as  a  time  series  of  ‘radio-frequency’  (RF)  data  images, 
while,  simultaneously,  tissue  is  externally  vibrated  with  a  broad¬ 
band  mechanical  excitation.  A  time  series  of  tissue  displacements 
or  strain  images  are  computed  from  consecutive  RF  data  images. 
The  tissue  displacement  as  a  function  of  time  at  a  given  spatial 
location  can  be  viewed  as  the  output  of  a  linear  system  whose  in¬ 
put  is  the  motion  of  the  exciter  as  a  function  of  time.  Therefore  a 
frequency  response  or  Transfer  Function  (TF)  that  relates  the  tissue 
motion  at  any  spatial  location  with  the  exciter  motion  as  a  refer¬ 
ence  can  be  computed  in  the  frequency  domain.  Alternatively,  if 
the  exciter  motion  is  not  measured,  a  tissue  region,  typically  in 
the  focal  area  of  the  ultrasound  beam,  can  be  selected  as  the  refer¬ 
ence.  For  display,  the  change  in  the  transfer  functions  from  one 
spatial  location  to  another  can  be  computed,  for  example  as  the 
L2-norm  of  the  difference  between  the  transfer  functions  over  the 
vibration  frequency  range. 

In  Salcudean  et  al.  (2006)  we  introduced  prostate  ultrasound  vi¬ 
bro-elastography.  The  signal  processing  used  to  obtain  the  transfer 
function  images  used  in  this  paper  are  presented  in  detail  in  Salcu¬ 
dean  et  al.  (2006),  together  with  phantom  images  and  initial 
in  vivo  prostate  imaging  data  from  three  patients.  We  showed 
qualitatively  that  this  method  has  the  potential  to  improve  the  vis¬ 
ibility  of  the  prostate.  In  this  article,  our  goal  is  to  evaluate  quanti¬ 
tatively  the  ability  of  vibro-elastography  to  visualize  the  prostate 
in  comparison  to  the  commonly  used  B-mode  imaging. 

Toward  this  goal,  two  groups  of  evaluation  measures  are  pro¬ 
posed:  image-based  and  volume-based  measures.  We  first 
evaluate  the  quality  of  the  images,  by  computing  the  standard 
measure  of  ‘contrast-to-noise  ratio’  (CNR)  in  VE  and  B-mode 
images.  Since  the  CNR  measure  does  not  effectively  assess  edge 
quality,  we  also  analyze  the  quality  of  edges  based  on  edge 
strength  and  edge  continuity.  For  edge  strength,  we  use  a  gradi¬ 
ent-based  edge  filter  and  a  statistical  edge  detector.  For  edge  con¬ 
tinuity,  we  compute  the  similarity  of  adjacent  edge  points  using  a 
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Fig.  1.  Schematic  representation  of  vibro-elastography  imaging.  From  left  to  right,  a  broad-band  external  mechanical  excitation  x0(t)  is  applied  to  the  sample;  the  axial 
motion  Xjk(t )  of  a  tissue  block  indexed  axially  by  j  and  laterally  by  k  within  the  ultrasound  image  is  obtained  from  RF  images  at  time  t  and  t  -  At;  transfer  functions  from  a 
reference  to  each  of  the  tissue  blocks  are  estimated,  with  the  illustration  showing  the  reference  being  the  motion  xjk(t)  of  block  jk;  images  are  produced  by  displaying  the  L2- 
norm  of  the  difference  in  transfer  functions. 
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correlation-based  measure.  We  compare  the  edge  evaluation  re¬ 
sults  in  VE  vs.  B-mode  images.  Then,  in  order  to  assess  whether 
the  delineated  prostate  boundary  in  VE  images  is  indeed  the  pros¬ 
tate,  we  use  volume-based  measures  to  compare  the  overall  shape 
and  size  of  the  gland  as  seen  in  VE  and  B-mode  ultrasound  images, 
with  MR  images  as  the  gold  standard. 

The  quality  assessment  criteria  used  in  this  paper  are  based  on 
both  standard  measures  (percentage  volume  difference,  percent¬ 
age  volume  error,  CNR,  gradient  filter  results)  and  on  new  mea¬ 
sures,  such  as  changes  in  image  statistics  at  edges  and  edge 
continuity.  These  new  measures  had  to  be  developed  to  deal  with 
the  particular  situation  of  prostate  segmentation,  which  requires 
the  identification  of  a  thin  capsule  within  a  background  of  rela¬ 
tively  uniform  echogenicity. 

A  preliminary  version  of  the  results  presented  in  this  paper, 
however  with  limited  details  and  analyses,  and  fewer  patients, 
has  appeared  in  Salcudean  et  al.  (2009)  and  Mahdavi  et  al.  (2009). 

The  paper  is  organized  as  follows:  In  Section  2  we  summarize 
the  patient  data  acquisition  and  the  vibro-elastography  imaging 
method  used  in  this  study.  In  Section  2.1,  our  data  acquisition 
and  the  resulting  images  are  described.  The  proposed  evaluation 
measures  of  this  paper  are  presented  in  Section  2.2.  The  results 
of  applying  these  measures  to  the  collected  MRI,  B-mode  ultra¬ 
sound  and  VE  data  are  described  in  Section  3.  Finally,  conclusions 
and  a  discussion  of  our  results  are  found  in  Section  4,  which  also 
provides  avenues  for  future  research. 


2.  Materials  and  methods 

2.1.  Data  acquisition 


Fig.  2.  System  setup  for  vibro-elastography  data  acquisition. 
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The  B-mode  ultrasound,  VE,  and  MR  images  used  in  this  paper 
were  acquired  from  patients  going  through  the  standard  LDR  pros¬ 
tate  brachytherapy  procedure  at  Vancouver  Cancer  Center,  BC  Can¬ 
cer  Agency,  or  radical  prostatectomy  at  the  Vancouver  General 
Hospital.  The  institutional  ethics  approval  and  informed  patient 
consent  were  obtained  prior  to  data  collection. 

The  MR  images  were  collected  between  one  and  two  weeks 
prior  to  treatment.  T2 -weighted  transverse  MR  images  (slice  spac¬ 
ing  4  mm,  pixel  size  0.27  mm  x  0.27  mm)  were  collected  at  the 
UBC  MRI  research  centre,  Vancouver,  with  a  Philips  Medical  Sys¬ 
tems  Achieva  3.0  Tesla  MRI  scanner.  To  minimize  the  deformation 
of  the  gland  and  for  patient  comfort,  imaging  was  carried  out  in  a 
supine  position  and  a  cardiac  coil  was  used. 

The  ultrasound  data  were  collected  intra-operatively,  just  prior 
to  the  actual  brachytherapy  or  prostatectomy  intervention,  using 
the  hardware  described  in  Section  2.1.1  and  the  signal  processing 
described  in  Section  2.1.2. 

2AA.  VE  system  hardware 

Fig.  2  shows  the  transrectal  ultrasound  (TRUS)  VE  actuation  sys¬ 
tem  developed  in  our  research  group  for  VE  imaging  of  the  pros¬ 
tate.  An  LDR  brachytherapy  stepper  (EXII,  CIVCO  Medical 
Solutions)  was  modified  to  enable  acquisition  of  RF  data  volumes. 
The  rotation  of  the  cradle  was  motorized  to  enable  smooth  rotation 
of  the  TRUS  transducer  and  a  shaker  was  mounted  on  the  trans¬ 
ducer  cradle  in  order  to  radially  vibrate  the  TRUS  probe.  Fig.  3  dis¬ 
plays  a  close-up  view  of  the  motorized  cradle.  A  PC-based  control 
interface  allows  the  user  to  control  the  rotation  motion  range 
(-45°  to  50°,  0°  being  the  mid-sagittal  plane),  the  amplitude,  and 
frequency  range  of  the  transducer  vibration.  Synchronized  with 
the  motion  of  the  probe,  frames  of  RF  data  are  collected  from  a  Son- 
ixRP  ultrasound  machine  with  the  sagittal  array  of  a  dual-plane  lin- 
ear/microconvex  broad-band  5-9  MHz  endorectal  transducer 
(Ultrasonix  Medical  Corp.).  The  RF  sampling  frequency  is  20  MHz. 


Fig.  3.  Details  of  the  motorized  cradle. 


The  vibrating  TRUS  probe,  with  a  vibration  range  of  2-1 0  Hz  (for 
the  first  14  cases)  and  2-5  Hz  (for  the  last  six  cases)  and  vibration 
amplitude  of  approximately  1  mm,  was  swept  from  -45°  to  50° 
covering  a  fan  of  longitudinal  planes  passing  through  the  trans¬ 
ducer  axis  with  a  predefined  angular  separation  of  2°.  At  each  an¬ 
gle,  one  B-mode  image  and  frames  of  RF  data  were  collected 
continuously  for  2  s  at  an  approximate  frame  rate  of  40  fps.  Imag¬ 
ing  depth  was  set  to  5  cm  (or  6  cm  for  large  prostates).  Fig.  4  dis¬ 
plays  the  orientation  of  the  fan  of  longitudinal  planes  at  which 
data  is  collected,  with  respect  to  the  prostate  and  the  TRUS  probe. 
These  planes,  which  we  will  simply  refer  to  as  ‘longitudinal’,  pass 
through  the  transducer  axis  and  include  the  mid-sagittal  plane  at 
angle  0°. 


Fig.  4.  An  illustration  of  the  longitudinal  planes  at  which  data  is  collected,  with 
respect  to  the  prostate  and  TRUS  probe.  Here,  the  mid-sagittal  plane,  at  0°,  and  two 
other  planes  at  angles  of  -20°  and  +10°  are  shown.  The  orientation  of  a  transverse 
plane  in  the  mid-gland  is  also  displayed. 
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2  A  2.  VE  system  signal  processing 

The  RF  data  were  processed  to  obtain  longitudinal  VE  transfer 
function  images.  Single  DOF  axial  motion  estimation  was  used  in 
processing  the  RF  data.  Each  RF  data  frame  has  128  lines  of  RF  data, 
each  having  1296  samples  (1424  for  larger  prostates).  The  col¬ 
lected  RF  data  lines  are  split  into  blocks  of  26  samples,  or  equiva¬ 
lently  1  mm,  with  50%  block  overlap.  The  axial  displacement  Xjk{t ) 
of  each  block  at  axial  location  k  and  lateral  location  (line)  j,  and 
subsequently  the  axial  strain,  were  computed  from  one  RF  data 
frame  to  the  next  by  using  a  correlation-based  method,  as  de¬ 
scribed  in  Zahiri-Azar  and  Salcudean  (2006),  resulting  in  axial  dis¬ 
placement  images  of  128  lines  by  100  blocks. 

An  image-based  reference  was  used  to  compute  the  transfer 
function  images.  It  was  computed  as  the  average  strain,  at  half 
the  tissue  imaging  depth,  of  all  the  128  lines.  This  specific  depth 
was  chosen  due  to  two  reasons.  First,  the  ultrasound  imaging  focal 
point  is  usually  set  at  this  depth,  resulting  in  a  more  accurate  mo¬ 
tion  estimation.  Second,  in  our  images,  the  half  depth  line  is  more 
likely  to  be  enclosed  entirely  within  the  prostate,  resulting  in  more 
uniform  mechanical  properties  along  the  line. 

The  transfer  functions  (EP*f(j2nf))  from  this  reference  to  each 
of  the  blocks  in  the  strain  images  were  computed  using  standard 
signal  processing  methods  described  in  Salcudean  et  al.  (2006). 
Transfer  function  images  were  generated  by  computing: 

TFjk  =  JJ*  (h^(/2tt/) -  rf^-k(j2nf))2df  (1 ) 

where  [/i,/2]  describes  the  frequency  range  of  interest,  which  in  this 
paper  coincides  with  the  range  of  the  broad-band  vibration  applied 
to  tissue. 

If  we  assume  that  the  Fourier  transform  of  the  reference  de¬ 
scribed  above  is  unity,  then  the  contrast  in  the  transfer  function 
image  from  one  spatial  location  (e.g.  a  reference)  to  another  corre¬ 
sponds  to  the  strain  energy  difference  between  these  locations  in 
the  frequency  range  to  f2.  By  computing  the  ‘difference’  between 
transfer  functions  of  consecutive  blocks  j  and  j  -  1  on  line  k,  with 
respect  to  reference  re/ ( Eq.  (1)),  changes  in  stiffness,  including 
the  prostate  boundary,  are  highlighted.  Throughout  this  paper, 
we  will  refer  to  these  vibro-elastography  transfer  function  images 
as  VE  images.  An  approach  to  obtain  both  elasticity  contrast  and 
relaxation-time/viscosity  contrast  using  a  Voigt  tissue  model  to 
represent  the  transfer  functions  is  presented  in  Eskandari  et  al. 
(2008). 


From  these  sets  of  longitudinal  images,  3D  VE  volumes  were 
generated  by  interpolation  (slice  spacing  in  transverse  direction, 
0.43  mm,  pixel  size  0.5  mm  x  0.5  mm).  Similarly,  the  3D  B-mode 
volume  (slice  spacing  in  transverse  direction  0.43  mm,  pixel  size 
0.37  mm  x  0.37  mm)  was  created  from  the  collected  longitudinal 
B-mode  images.  Various  approaches  to  creating  3D  volumes  are 
described  in  detail  in  Fenster  et  al.  (2001).  Our  method  of  con¬ 
structing  a  3D  volume  by  interpolating  longitudinal  images  suffers 
from  a  decrease  in  image  resolution  as  the  depth  increases.  How¬ 
ever,  the  resolution  can  be  increased  by  using  finer  angle  incre¬ 
ments  at  the  expense  of  scan  time.  Another  alternative  is  to  use 
the  transverse  (convex)  array  of  the  TRUS  probe  to  generate  a  3D 
sector  view  by  changing  the  depth  of  the  transducer  with  respect 
to  the  prostate.  This  approach  also  has  the  issue  of  decreasing  res¬ 
olution  at  increased  depth  because  the  scan  is  acquired  with  a  con¬ 
vex  array.  Furthermore,  such  a  transverse  plane  sweep  changes  the 
TRUS  position  in  the  rectum  causing  the  prostate  to  move  as  a 
function  of  probe  depth,  which  is  clearly  undesirable.  The  analysis 
performed  in  this  paper  uses  the  transverse  plane  of  the  con¬ 
structed  3D  volumes  for  evaluation.  This  choice  was  made  because 
of  the  preference  of  the  transverse  view  by  the  clinicians,  and  also 
for  agreement  with  the  transverse  MRI  images. 

Fig.  5  shows  transverse  VE,  B-mode  and  MRI  images  of  the  pros¬ 
tate  mid-gland  of  two  patients.  Half  of  the  prostate  boundary  is 
delineated  in  one  of  the  image  sets.  In  Fig.  6  mid-sagittal  VE  (top 
row)  and  B-mode  (bottom  row)  prostate  images  of  three  different 
patients  can  be  seen.  The  prostate  is  the  stiffer  region  as  outlined  in 
the  VE  image. 

Table  1  describes  the  image  types  and  number  of  cases  used  in 
this  paper.  VE  and  B-mode  images  were  obtained  from  20  patients. 
Among  these  20  cases,  which  consist  of  178  co-registered  VE  and 
B-mode  image  pairs,  107  images  are  in  the  transverse  plane  and 
are  used  for  edge  evaluation.  The  remaining  71  images  are  in  the 
longitudinal  plane  at  multiples  of  5°  intervals  with  the  mid-sagittal 
plane  and  are  used  along  with  the  107  transverse  images,  for  CNR 
analysis.  MRI  data  was  also  available  for  nine  of  the  patients.  These 
nine  data  sets  are  used  for  volume  evaluation.  The  initial  study 
started  by  recruiting  patients  for  VE  imaging  at  the  time  of  the  vol¬ 
ume  examination.  However,  we  found  that  the  patients  were  ner¬ 
vous  and  very  uncomfortable  with  the  additional  time  required  to 
take  the  images  with  a  different  ultrasound  machine.  In  some  cases 
we  were  able  to  collect  US  data,  but  the  same  patients  were  not 
available  for  MRI,  due  to  illness,  ineligibility  for  MRI,  and  difficulty 
traveling  for  the  additional  medical  exam.  For  the  data  acquired 


Fig.  5.  Transverse  B-mode  (left),  VE  (middle)  and  MRI  (right)  prostate  images  of  two  patients.  The  boundary  of  the  prostate  is  partially  segmented  in  the  second  set  of  images. 
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Fig.  6.  VE  (top)  and  B-mode  (bottom)  sagittal  images  of  the  prostate  of  three  different  patients.  The  boundary  of  the  prostate  is  outlined  in  one  of  the  patients. 


Table  1 

Description  of  the  data  used  in  this  paper. 


Data  set  size 

Image  type 

Image  plane 

Volume  analysis 

9  cases 

MRI,  VE, 
B-mode 

transverse 

CNR  analysis 

178  images 
(20  cases) 

VE,  B-mode 

107  transverse, 

71  longitudinal 

Edge  analysis 

107  images 
(11  cases) 

VE,  B-mode 

transverse 

during  the  volume  study,  the  acquisition  time  had  to  be  mini¬ 
mized.  Therefore  the  number  of  MRI  scans  that  we  have  used  in 
our  study  is  limited.  Nevertheless,  this  number  is  similar  to  that 
used  in  Smith  et  al.  (2007)  that  compares  B-mode  based  prostate 
segmentation  to  MRI.  Furthermore,  in  Smith  et  al.  (2007)  and  most 
of  the  studies  cited  therein,  post-operative  MRI  is  compared  with 
TRUS.  However,  tissue  will  change  as  a  result  of  treatment.  We  be¬ 
lieve  that  comparison  based  on  pre-operative  images  is  necessary 
for  general  assessment  of  the  gland’s  visibility  and  treatment 
planning. 

2.2.  Evaluation  methods 

In  order  to  assess  the  visibility  of  the  prostate  in  VE  images,  we 
utilize  several  evaluation  measures.  These  include  image-based 
measures,  which  consist  of  contrast-to-noise  ratio,  edge  continuity 
and  edge  strength,  and  volume-based  measures,  which  consist  of 
volume  error  and  volume  difference.  For  more  accurate  region- 
based  evaluation,  the  results  of  these  measures  will  be  reported 
for  nine  sectors  of  the  gland.  To  do  so,  as  shown  in  Fig.  7,  the  pros¬ 
tate  is  divided  into  the  base,  mid,  and  apex  regions.  Then,  each  re¬ 
gion  is  further  divided  into  posterior,  anterior  and  lateral  parts. 
Quadrant-based  division  of  the  prostate  is  a  common  approach  in 
the  literature  (Sidhu  et  al.,  2002;  Thomas  et  al.,  2007).  However, 
for  our  application,  this  subdivision  scheme  provides  more  detailed 
reporting  of  results  for  different  regions  of  the  gland.  This  is  moti¬ 
vated  by  the  difference  in  the  importance  of  the  prostatic  and  per¬ 
iprostatic  tissue  in  various  regions.  For  example,  various  clinical 
studies  have  shown  that  in  LDR  brachytherapy,  overdosing  the 


posterior  region  of  the  prostate  results  in  rectal  complications 
(Snyder  et  al.,  2001). 

2.2.1.  Image-based  measures 

Accurate  delineation  of  the  prostate  is  affected  by  how  the  pros¬ 
tate  is  visualized.  A  “good”  image  of  the  prostate  is  one  in  which 
the  prostate  has  high  contrast  relative  to  the  background  and  has 
well-defined  edges.  A  good  edge  should  be  both  “continuous” 
and  “strong”.  In  this  analysis,  VE  images  of  the  prostate  are  com¬ 
pared  to  B-mode  ultrasound  images  using  the  measures  of  con¬ 
trast,  edge  strength,  and  edge  continuity  as  described  below. 

2.2.2.  The  contrast-to-noise  ratio 

To  compare  the  contrast  of  VE  and  B-mode  images,  the  contrast- 
to-noise  ratio  (CNR)  was  calculated  using  the  following  equation 
(Bilgen  and  Insana,  1997): 

CNR  =  2(mrmb)2 

o2t+a2b 

in  which  m  and  a2  are  the  mean  and  variance  of  pixel  intensities  of 
the  target,  t,  and  background,  b,  in  a  region  of  interest  (ROI).  The  ROI 
of  the  target  and  background  were  manually  selected  in  four  re¬ 
gions  of  the  prostate  (lateral  left,  lateral  right,  anterior  and  poster¬ 
ior).  The  target  is  an  area  inside  the  prostate  while  the  background 
is  an  area  outside  the  prostate  close  to  the  target  ROI  (respectively 
yellow1  and  blue  boxes  in  Fig.  8).  The  physical  size  and  position  of 
the  ROI  in  VE  and  B-mode  images  were  similar.  Additionally,  to  en¬ 
sure  similarity  between  the  intensity  range  in  both  set  of  images, 
histogram  stretching  [0-255]  was  initially  applied  to  the  images. 

The  contrast  between  the  prostate  and  its  background  is  typi¬ 
cally  low  in  B-mode  ultrasound  images  (see  Fig.  5).  However,  due 
to  the  presence  of  a  visible  edge,  delineation  of  the  prostate  is  still 
possible.  Therefore,  CNR  alone  can  not  evaluate  the  visibility  of  an 
object  in  an  image  and  edge  evaluation  is  also  required. 

2.2.3.  Edge  strength 

An  edge  can  be  defined  as  the  boundary  between  two  regions  in 
an  image  that  are  different  from  each  other  with  respect  to  some 


1  For  interpretation  of  color  in  Figs.  1-3,  8, 9  and  13-15,  the  reader  is  referred  to  the 

web  version  of  this  article. 
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Ant. 


Fig.  7.  Division  of  the  prostate  into  nine  sectors. 


(a)  B-mode  ultrasound  (b)  VE 


Fig.  8.  Selection  of  inside  (yellow  boxes)  and  outside  (blue  boxes)  regions  of  the  prostate  for  CNR  computation  in  (a)  B-mode  and  (b)  VE  images.  (For  interpretation  of  the 
references  to  colour  in  this  figure  legend,  the  reader  is  referred  to  the  web  version  of  this  article.) 


local  property  (Kitchen  and  Rosenfeld,  1981).  While  conceptually 
clear,  this  definition  is  difficult  to  quantify.  However,  for  the  pur¬ 
pose  of  this  study  we  need  quantitative  measures.  Two  different 
measures  of  edge  strength  are  presented  here.  One  is  a  conven¬ 
tional  gradient-based  edge  filter  and  the  second  one  is  a  test  that 
identifies  the  statistical  changes  in  image  intensity.  The  edge 
strength  measures  are  computed  for  radial  edge  profiles  formed 
as  follows:  On  each  transverse  image,  we  extended  radii  at 
Oi  =  i  x  30°,  i  =  1, . . . ,  12  in  polar  coordinates  originating  from  a 
manually  selected  center  point,  C,  inside  the  prostate.  The  intersec¬ 
tions  r0. ,  i  =  1, . . . ,  12,  of  these  radii  with  the  prostate  boundary 
were  manually  identified  for  each  angle  0/.  For  each  edge  point,  a 
radial  edge  intensity  profile  J0.(r)  was  extracted.  The  measures  of 
edge  strength  were  calculated  for  the  window  of  r  e  [r0i  -  Ad, 


r0i  +  Ad],  where  Ad  is  half  the  length  of  the  edge  intensity  profile 
(see  Fig.  9). 

Gradient-based  measure  of  edge  strength :  The  following  gradient 
formulation,  also  used  in  Abolmaesumi  and  Sirouspour  (2004),  was 
used  as  the  edge  filter  acting  on  a  radial  edge  intensity  /(r),  where 
we  remove,  for  convenience,  the  Oi  index  of  I: 

fedgeir)  =  1/3  x  [J(r  +  2Ar)  +  I(r  +  Ar)  +  J(r)  -  t(r  -  A r)  -  I(r 

-  2Ar)  +  l(r  -  3A r)]2  (3) 

where  Ar  is  the  physical  size  of  the  image  pixel. 

M,  our  measure  of  edge  strength,  is  the  sum  of  the  filter  outputs 
on  windows  of  size  n  pixels  (n  an  odd  integer)  normalized  to  the 
sum  of  edge  filter  values  on  the  entire  edge  profile: 


(a)  (b) 

Fig.  9.  (a)  The  radii  used  for  edge  profile  extraction  in  a  VE  image,  originating  from  C  and  with  angles  0,-  =  i  x  30°,  i  =  1, ....  12.  For  one  of  the  radii,  the  two  neighboring  radii 
are  also  illustrated,  (b)  A  close-up  view  of  one  of  the  rays  used  for  extracting  the  edge  profile  /0j(r).  The  measures  of  edge  strength  are  calculated  for  the  window  of 
r  e  [r0j  -  Ad,  r0.  +  Ad],  where  r0.  is  a  manually  selected  edge  point  along  the  ray.  The  neighboring  edge  profiles  I0j±so,  are  extracted  similarly  and  used  along  with  I0{  in  the  edge 
continuity  measure. 
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j=r+(n-l)Ar/2  /  j=r0.+Ad 

M(r)  =  ]T  fedge (j)  /  E  fedge (?)  (4) 

j=r-(n-l)Ar/2  /  j=rg.-Ad 

The  normalization  is  performed  so  that  we  can  compare  results 
for  different  areas.  M  is  computed  along  r  e  [r0f  -  Ad,  re.  +  Ad],  and 
is  expected  to  have  a  strong  peak  at  the  edge.  In  our  implementa¬ 
tion,  parameter  values  were  set  to  Ad  =  0.5  cm  and  n  =  5  pixels. 

DF  statistical  test :  The  performance  of  the  gradient-based  edge 
detector  can  be  plagued  by  local  minima  in  US  images.  Therefore, 
we  present  a  new  approach  that  models  the  difference  of  the  radial 
edge  intensity  profile  as  an  autoregressive  process.  The  edge 
strength  is  characterized  based  on  the  degree  of  stationarity  of  this 
process.  The  stationarity  of  the  edge  profile  is  tested  using  the 
statistical  test  proposed  by  Dickey  and  Fuller  (1979).  Each  radial 
intensity  profile,  was  considered  as  a  time  series  Ig.  (k)  :  =  I0i 
(kAr),  where  the  discretized  radial  distance  k,  such  that 
kAr  e  [re.  -  Ad,  re.  +  Ad],  replaces  the  usual  time  index. 

The  edge  profiles,  / 0.(k),  can  be  modeled  as  a  first  order  autore¬ 
gressive  AR(1)  processes  as  follows: 


Iei(k)  =  phi(k-  l)  +  er  (5) 

where  p  is  a  real  number  and  er  is  a  sequence  of  independent  nor¬ 
mal  variables  with  mean  0  and  variance  a2.  In  order  to  show  that  an 
AR(1)  model  is  sufficient  for  modeling  the  edge  profiles,  we  com¬ 
puted  the  partial  autocorrelation  function  (PACF)  of  /0.(k).  The  PACF 
of  an  AR(1)  process  has  significant  values  only  at  lag  =  1.  In  78%  of 
the  edge  profiles  extracted  from  both  B-mode  and  VE  images,  at 
the  significance  level  of  0.05,  the  PACF  function  only  has  significant 
values  at  lag  =  1 . 

I0i  (k)  is  stationary  if  \p\  <  1.  If  a  unit  root  exists  (\p\  =  1),  then  the 
variance  of  I0i(k)  is  rcr2,  and  therefore  I0i{k)  is  non-stationary.  In 
many  economics  applications,  the  existence  of  the  unit  root,  which 
is  an  indication  of  a  “trend”  or  a  “shock”,  is  important  for  modeling 
and  forecasting  the  future  observations  of  a  time  series.  Dickey  and 
Fuller  (1979)  provided  a  statistical  method  to  test  an  AR  model  for 
the  null  hypothesis  of  the  existence  of  a  unit  root.  If  we  re-write  (5) 
as  follows: 

Ale,(k)  =  Ie,(k)  -  h,(k  -  \) 

=  (p-  1)h,0<  -  1)  +  er 
=  yh,(k-  l)  +  er 

then  the  DF  test  is  formulated  as  follows: 

H0  :  p  =  1  <-  H0  :  y  =  0  (6) 

Hi  :  p  <  1  <-►  H0  :  y  <  0  (7) 

Note  that  the  test  is  performed  on  the  residuals  and  not  the 
time  series  samples.  Therefore,  the  standard  t-distribution  cannot 
be  used.  Dickey  and  Fuller  provide  a  non-standard  statistic  t, 
which  depends  on  the  number  of  observations,  and  provide  tables 
of  critical  values  for  it.  In  other  words,  based  on  the  calculated  va¬ 
lue  of  t,  they  provide  the  significance  level  at  which  the  null 
hypothesis  above  can  be  rejected.  We  used  the  implementation  re¬ 
ported  in  Kanzler  (1998)  to  perform  the  DF  test. 

The  statistical  properties  of  the  intensity  profile  are  altered  at 
the  edge.  The  existence  of  a  strong  edge  in  the  time  series  is  an 
obvious  trend.  A  signal  with  a  trend  cannot  be  stationary,  since 
its  statistical  moments  depend  on  time,  or  in  our  model,  on  dis¬ 
tance.  In  other  words,  if  one  traces  the  image  in  the  radial  direc¬ 
tion,  the  intensity  profile  tends  to  become  non-stationary  upon 
passing  through  an  edge.  To  evaluate  the  edge  quality,  we  compute 
and  report  the  percentage  of  edge  profiles  for  which,  according  to 
the  DF  test,  the  stationarity  hypothesis  is  strongly  rejected 
(p  <  0.05)  for  the  edge  profile.  For  an  image  with  strong  edges, 
one  expects  to  see  a  high  percentage  of  edge  profiles  for  which 


the  unit  root  exists  and  the  stationarity  hypothesis  is  strongly 
rejected. 


2.2.4.  Edge  continuity 

Detecting  an  edge  point  on  a  single  edge  profile  can  not  guaran¬ 
tee  the  presence  of  an  edge.  Continuity  of  the  presence  of  such  a 
point  within  a  neighborhood  is  an  important  factor  for  the  visibil¬ 
ity  of  the  edge.  We  measure  this  continuity  by  measuring  the  sim¬ 
ilarity  of  the  edge  region  within  a  neighborhood. 

For  a  continuous  edge,  two  neighboring  edge  profiles  are  ex¬ 
pected  to  be  similar,  although  slight  differences  may  be  present 
due  to  image  noise  or  in  the  case  of  ultrasound,  speckle.  The  nor¬ 
malized  cross-correlations  Rougt±se(r)  of  the  two  neighboring  edge 
intensity  profiles  l0{  (r)  and  I0i±sg(r),  r  e  [r0.  -  Ad,  r0.  +  Ad]  and  the 
average  c(0,-)(r)  =  \  [R0i,oi+se  +  Ro^-se]  were  calculated  for  each 
point,  r0.,  at  which  the  radius  r0.  intersects  the  edge  (see  Fig.  9). 
The  parameters  we  used  for  this  implementation  were  0*  =  i  x  30°, 
i  =  1, . . . ,  12,  Ad  =  0.2  cm,  and  SO  such  that  the  arc  length  between 
the  two  adjacent  edge  profiles  is  0.2  cm. 

For  a  large  similarity  between  adjacent  edge  profiles,  c6i  should 
have  a  shape  similar  to  the  shape  of  a  Gaussian  function  with  large 
peak  at  the  edge  point  and  a  small  standard  deviation.  Thus,  we 
propose  that  the  following  edge  continuity  measure,  K(0i),  be 
calculated: 


(8) 


in  which  P(0/)  and  a (0f)  are  the  peak  and  standard  deviation  of  a 
Gaussian  function  fitted  to  c0{. 


2.2.5.  Volume-based  measures 

The  volume  of  the  prostate  is  an  important  parameter  in  plan¬ 
ning  the  dose  distribution  in  LDR  brachytherapy.  A  well-defined 
3D  shape  can  also  aid  prostatectomy.  To  evaluate  the  shape  and 
size  of  the  prostate  created  from  VE  images,  we  compare  them 
with  the  shape  and  size  extracted  from  the  commonly  used  B- 
mode  ultrasound  images.  Since  MR  images  of  the  prostate  provide 
more  anatomical  details  including  visualizing  the  boundaries,  the 
3D  surface  extracted  from  MR  images  is  used  as  the  gold  standard. 
Contouring  in  all  three  image  types  was  performed  manually  and 
by  three  observers:  one  radiation  oncologist  and  two  trained  by  ex¬ 
perts.  The  ‘volume  difference’  and  ‘volume  error’  between  surfaces 
created  from  MR  and  VE/B-mode  images  (i.e.  MR  vs.  VE  or  MR  vs. 
B-mode)  provide  shape  and  size  similarity  errors  that  can  be  used 
to  compare  to  the  gold  standard  provided  by  MRI. 

The  percent  volume  difference  provides  a  measure  of  the  differ¬ 
ence  in  the  size  of  the  total  gland  and  each  of  the  nine  sectors  in 
VE/B-mode  images  compared  to  the  gold  standard  MRI.  It  is  de¬ 
fined  as: 

^diff%  —  100  X  (VvE/B-mode  ~  ^MRl)/^MRI  (9) 

The  percent  volume  error  is  the  volume  of  the  non-overlapping 
region  between  the  surface  of  the  prostate  generated  from  VE/B- 
mode  images  and  that  of  MR,  divided  by  the  sum  of  the  two  vol¬ 
umes.  This  is  defined  as: 


Verr%  —  100  X  |  ^VvE/B-mode  T  Vmri  ^( ^VE/B-mode P| MRI ))  (VyE/B-mode  +  Vmri) 

(10) 

In  other  words,  it  provides  a  measure  of  the  difference  in  the 
shape  of  the  gland  compared  to  the  gold  standard.  This  definition 
is  equivalent  to  1  -  DSC,  in  which  DSC  is  the  Dice  Similarity  Coef¬ 
ficient  (Dice,  1945). 

The  3D  shape  of  the  prostate,  in  each  modality,  was  extracted  by 
manually  segmenting  the  2D  images  with  the  use  of  the  Stradwin 
(Treece  et  al.,  2000)  software.  This  software  was  then  used  to 
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Fig.  10.  Registration  of  B-mode/VE  prostate  surfaces  to  MRI. 


Table  2 

CNR  comparison  of  VE  and  B-mode  images. 


Base 

Mid 

Apex 

CNR  VE 

10.25  ±  12.83 

13.73  ±5.85 

20.51  ±23.13 

CNR  B-mode 

2.07  ±  1.06 

1.43  ±0.75 

1.56  ±  1.16 

transform  the 

2D  contours  into 

a  3D  surface 

which  could  be 

opened  in  MATLAB®.  MATLAB®  is  the  environment  in  which  we 
performed  all  our  calculations. 

To  register  the  MRI  and  ultrasound  images  we  opted  for  a  rigid 
registration  approach,  as  opposed  to  a  deformable  registration  one. 
Indeed,  these  images  were  not  acquired  at  the  same  time  and  the 
patients  were  in  the  dorsal  lithotomy  position  during  B-mode 
and  VE  acquisition  and  supine  position  during  MR  acquisition.  Fur¬ 
thermore,  the  patients  were  anaesthetized  and  relaxed  prior  to  the 
procedure,  while  they  were  awake  and  possibly  tense  during  the 
MRI  exam.  A  non-rigid  (deformable)  registration  is  not  a  suitable 
registration  option  since  the  goal  is  to  understand  the  differences 
between  the  manually  segmented  surfaces.  A  deformable  registra¬ 
tion  which  maps  one  surface  to  the  other  will  conceal  these 
differences. 

We  assume  that  the  main  cause  of  mis-registration  of  the  sur¬ 
faces  is  the  angle  of  the  prostates’  base-apex  axis  with  the  TRUS 
probe.  This  is  mainly  a  result  of  the  patient’s  orientation  during 


edge  filter  values  in  B-mode  vs.  VE 


Fig.  12.  The  values  of  the  gradient-based  edge  strength  measure  (M)  along  the  edge 
profiles  for  VE  and  B-mode  images.  Error  bars  represent  the  inter-patient  standard 
deviation  of  the  M  values. 


Table  3 

The  percentage  of  non-stationary  prostate  edge  profiles  in  different  areas  of  the  B- 
mode  and  VE  images.  Standard  deviations  reported  characterize  inter-patient 
variations. 


% 

VE 

B-mode  ultrasound 

Base 

Mid 

Apex 

Base 

Mid 

Apex 

Ant. 

Lat. 

Post. 

67.9  ±  4.6 

79.5  ±3.1 

78.6  ±3.5 

99.2  ±  1.1 

98.8  ±  0.9 
99.2  ±  1.9 

99.9  ±  0.2 

98.1  ±1.7 
99.8  ±  0.2 

71.4  ±2.8 

66.1  ±3.5 
67.9  ±3.2 

80.8  ±4.1 

81.3  ±2.5 
80.8  ±4.1 

76.6  ±7.5 

88.9  ±5.1 
84.1  ±8.1 

imaging.  We  also  assume  that  the  size  of  the  prostate  does  not 
change  between  data  collection  sessions  and  the  prostate  does 
not  rotate  around  its  base-apex  axis  by  a  large  angle.  For  each  sur¬ 
face,  this  axis  was  obtained  by  fitting  a  line  to  the  centers  of  the 
prostate  contours  in  each  image  slice. 

With  these  assumptions,  to  register  the  prostate  surfaces  PB  (the 
VE  or  B-mode  surface)  to  PA  (the  MRI  surface),  a  translation  tAB  is 
first  applied  in  order  to  match  their  geometric  centers  CA  and  CB. 
A  rotation  R0  around  the  matched  geometric  center  is  then  applied 
to  rotate  the  base-apex  axis  of  the  translated  prostate  PB  to  the 


base-anterior 


mid-anterior 


0.5 


0.5  -0.5 

distance  from  the  edge  point  (cm)  distance  from  the  edge  point  (cm)  distance  from  the  edge  point  (cm) 


Fig.  11.  The  values  of  the  normalized  gradient-based  edge  strength  measure  (M)  in  arbitrary  units  vs.  the  distance  from  the  edge  in  cm,  separated  for  the  nine  regions  of  the 
prostate  gland. 
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main  axis  of  PA.  6  is  the  angle  between  the  base-apex  axis  of  the 
two  surfaces,  obtained  as  described  above.  Finally  a  six  degree- 
of-freedom  Iterative  Closest  Point  (ICP)  method  (Besl  and  McKay, 
1992)  translates  (tiCp)  and  rotates  (RiCp)  the  resulting  surface  to 
fine-tune  the  registration,  resulting  in  Pe-registered  (Fig.  10),  is  de¬ 
scribed  as: 

^-registered  =  ^ICP  [Re  (Pb  +  ?4b)  +  ?ICp]  (H) 

The  volume  error  and  volume  difference  is  calculated  for  the 
rigidly  registered  surfaces. 

3.  Results 

3.1.  Image  comparison 

3.1.1.  The  contrast 

The  CNR  of  VE  and  B-mode  images,  averaged  over  the  three  sec¬ 
tions  of  the  prostate,  is  shown  in  Table  2.  Data  from  178  images 
(from  20  patients)  were  used  in  this  analysis.  In  all  three  regions 
the  CNR  of  VE  is  significantly  higher  than  that  of  B-mode 
(p  <  0.05  for  base,  and  p  <  0.001  for  the  mid-gland  and  apex 
regions). 

3.1.2.  Edge  strength:  gradient-based  measure 

The  gradient-based  measure  of  edge  strength,  M,  was  computed 
for  edge  profiles  in  the  nine  regions  described  in  Section  2.2 
(Fig.  7).  The  VE  and  B-mode  images  have  different  resolutions.  This 
is  due  to  the  fact  that  the  window  size  used  for  displacement  esti¬ 
mation  in  VE  images  is  larger  than  the  B-mode  pixel.  Therefore,  the 
choice  of  n  (the  window  size),  affects  the  value  computed  for  M. 
We  examined  values  of  window  sizes  in  the  range  of  n  =  2  to 
n  =  10  and  in  all  cases,  the  results  of  edge  analysis  were  consistent. 
The  reported  results  were  acquired  for  n  =  5.  The  result,  extracted 
from  107  images  of  11  patient  datasets,  is  illustrated  in  Fig.  11 
for  the  nine  regions  and  in  Fig.  12  for  all  regions  combined.  As  illus¬ 
trated  in  Fig.  1 1 ,  the  VE  edges  are  notably  stronger  than  B-mode 
edges  with  the  exception  of  the  anterior  base  and  the  posterior 
apex.  The  other  observation  of  note  is  that  the  values  of  M  show 
an  evident  peak  in  all  areas  of  VE  images,  with  the  exception  of 


the  anterior  base.  In  the  case  of  B-mode  images,  in  some  areas  such 
as  the  lateral  and  anterior  apex  and  the  lateral  mid-gland,  the 
edges  are  very  weak  and  are  not  represented  by  a  dominant  peak 
in  the  gradient  values.  In  the  posterior  region  of  the  prostate  (the 
last  row  in  Fig.  1 1 )  the  B-mode  edge  strength  appears  to  be  similar 
or  slightly  higher  than  the  VE  edge  strength. 

It  is  evident  from  Fig.  12  that,  overall,  edges  in  VE  images  are 
stronger.  At  the  edge  point,  the  normalized  M  value  is  2.2  times  lar¬ 
ger  in  VE  images  compared  to  B-mode  images. 

3.1.3.  Edge  strength:  DF  test  of  stationarity 

Table  3  shows  the  percentage  of  edge  profiles  for  which  the  DF 
test  strongly  shows  a  unit  root  (p  <  0.05),  and  therefore,  non-sta- 
tionarity.  107  images  from  11  patient  datasets  and  12  edge  profiles 
within  each  image,  were  used  in  this  analysis.  This  result  is  re¬ 
ported  separately  for  the  nine  regions.  The  edges  appear  stronger 
in  VE  images  in  all  areas,  with  the  exception  of  the  anterior  base 
(matching  the  result  from  the  gradient-based  edge  strength).  In 
general,  the  edges  in  the  base  region  are  relatively  weak  in  both 
modalities. 

3.1.4.  Edge  continuity 

Table  4  shows  the  edge  continuity  value,  I<,  computed  for  VE 
and  B-mode  images  in  nine  regions  of  the  gland.  107  images  from 
11  patients  were  used  in  this  analysis.  The  edge  continuity  of  VE  in 
all  regions  except  for  the  anterior  base  and  posterior  apex  is  shown 
to  be  superior  to  that  of  B-mode  images.  This  is  also  in  agreement 
with  the  results  of  the  gradient-based  measure  of  edge  strength 
(Fig.  11). 

3.2.  Volume  comparison 

Fig.  13  shows  an  example  of  VE  (magenta)  and  B-mode  (green) 
3D  surfaces  compared  to  that  of  MRI  (blue)  for  one  of  the  patients. 
Figs.  14  and  15  compare  the  percent  volume  error  and  volume  dif¬ 
ference  between  B-mode  and  MRI  and  between  VE  and  MRI  pros¬ 
tate  surfaces  manually  created  by  one  radiation  oncologist  (Figs.  14 
and  15b)  and  two  individuals  trained  by  experts  (Figs.  14, 15a  and 
Figs.  14,  15c).  Images  from  nine  patient  data  sets  were  used.  From 
Fig.  14  it  can  be  seen  that  in  most  regions  of  the  prostate,  the  mean 


Table  4 

Average  edge  continuity  measure,  I<,  for  the  nine  regions  of  the  gland. 


I<(9i)  VE 

K(6i)  B-mode  ultrasound 

Base 

Mid 

Apex 

Base 

Mid 

Apex 

Ant. 

0.64  ±  0.37 

0.87  ±  0.33 

0.84  ±  0.37 

0.71  ±0.52 

0.69  ±  0.41 

0.50  ±  0.32 

Lat. 

0.84  ±  0.39 

1.15  ±0.36 

0.85  ±  0.27 

0.63  ±  0.40 

0.48  ±  0.23 

0.39  ±  0.29 

Post. 

0.99  ±  0.44 

0.82  ±  0.35 

0.60  ±  0.55 

0.52  ±  0.38 

0.51  ±0.32 

0.75  ±  0.55 

Fig.  13.  Comparison  of  VE  (magenta)  vs.  MRI  (blue)  3D  surfaces,  on  the  left,  and  B-mode  (green)  vs.  MRI  (blue)  3D  surfaces,  on  the  right,  from  one  of  the  patients.  (For 
interpretation  of  the  references  to  colour  in  this  figure  legend,  the  reader  is  referred  to  the  web  version  of  this  article.) 
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(a)  Observer  1:  trained  by  expert 


(c)  Observer  3:  trained  by  expert 


Fig.  14.  A  comparison  between  VE  vs.  MRI  volume  error  and  B-mode  vs.  MRI 
volume  error,  showing  the  mean  and  inter-patient  standard  deviation  of  VeTT%  for 
three  observers.  Sample  data  points  are  also  shown  as  gray  dots. 


volume  error  between  VE  and  MRI  is  less  than  that  between 
B-mode  and  MRI. 


(a)  Observer  1:  trained  by  expert 


(b)  Observer  2:  Radiation  Oncologist 


(c)  Observer  3:  trained  by  expert 


Fig.  15.  A  comparison  between  VE  vs.  MRI  volume  difference  and  B-mode  vs.  MRI 
volume  difference,  showing  the  mean  and  inter-patient  standard  deviation  of  Vdiff% 
for  three  observers.  Sample  data  points  are  also  shown  as  gray  dots. 


Over  the  total  gland,  a  volume  error  of  8.8  ±  2.5%  for  VE  vs.  MRI 
and  10.3  ±4.6%  for  B-mode  vs.  MRI,  and  a  volume  difference  of 
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-4.6  ±  11.1%  for  VE  vs.  MRI  and  -4.1  ±17.1%  for  B-mode  vs.  MRI, 
averaged  over  nine  patients  and  three  observers,  were  obtained. 
However,  a  one-sided  analysis  of  variance  does  not  show  any  sta¬ 
tistically  significant  difference,  which  may  also  be  due  to  the  lim¬ 
ited  number  of  patients.  In  general  because  the  prostate  is  not 
discernible  at  the  base  and  apex,  a  larger  error  in  shape  (volume  er¬ 
ror)  and  size  (volume  difference)  between  VE  or  B-mode  surface 
and  the  gold  standard  in  these  two  regions  compared  to  the  mid¬ 
gland  is  expected.  However,  in  general,  and  over  the  three  observ¬ 
ers,  VE  errors  are  smaller  than  those  from  B-mode. 


4.  Discussion  and  conclusions 

In  this  paper  ultrasound  vibro-elastography  was  evaluated  as  an 
imaging  modality  for  the  visualization  of  the  prostate.  VE  transfer 
function  (TF)  images  were  qualitatively  and  quantitatively  com¬ 
pared  with  the  commonly  used  B-mode  ultrasound.  The  evaluation 
measures  used  were  both  image-based  (CNR,  edge  continuity  and 
edge  strength)  and  volume-based  (volume  error  and  volume  differ¬ 
ence).  A  nine  sector  analysis  was  used  for  more  detailed  character¬ 
ization.  The  results  on  178  images  suggest  that  the  VE  images  are 
significantly  superior  to  B-mode  images  in  terms  of  contrast  of  the 
gland.  The  gradient-based  and  the  DF-based  measures  of  edge 
strength  and  the  edge  continuity  measure,  on  107  images,  all  show 
that  on  average  VE  images  provide  stronger  edges  as  well.  It  is 
important  to  emphasize  that  the  statistically  significant  outcomes 
of  our  analysis  of  edge  continuity  and  gradient-based  edge 
strength  are  consistent:  the  edges  in  base-lateral,  mid-lateral, 
mid-anterior,  and  lateral  apex  are  significantly  stronger  and  more 
continuous  (p  <  0.05  for  the  one-sided  analysis  of  variance)  in  VE 
images  compared  to  B-mode  images.  The  I<  and  M  values,  in  addi¬ 
tion  to  the  visual  inspection  of  the  images,  show  that  on  average 
the  anterior  base  and  posterior  apex  of  the  prostate  has  weak  edges 
on  VE  images. 

The  B-mode  data  used  in  this  paper  was  collected  simulta¬ 
neously  with  the  RF  data  used  for  creating  the  VE  images.  In  other 
words,  B-mode  and  VE  images  were  acquired  under  exactly  similar 
conditions  and  are  co-registered. 

The  comparison  of  volumes  calculated  from  VE  and  B-mode 
prostate  images  show  that  VE  volumes  are  closer  to  the  MRI  gold 
standard  in  most  regions  of  the  prostate,  both  in  terms  of  shape 
and  size.  This  confirms  that  the  outlined  region  used  in  the  im¬ 
age-based  evaluation  of  VE,  is  indeed  the  prostate.  In  this  compar¬ 
ison,  prostate  images  from  9  patients  were  manually  delineated  by 
three  observers  (one  expert  and  two  trained  by  experts).  The  total 
gland  volume  error  for  VE  vs.  MRI  was  8.4  ±  2.9%,  8.3  ±  1.8%,  and 
9.8  ±  2.8%  for  the  three  observers.  For  B-mode  vs.  MRI,  these  values 
were  8.6  ±4.8%,  11.3  ±4.7%,  and  11.0  ±4.4%.  The  total  gland  vol¬ 
ume  difference  for  VE  vs.  MRI  was  -5.3  ±  11.7%,  -8.1  ±  8.8%,  and 
-0.3  ±12.1%  for  the  three  observers.  For  B-mode  vs.  MRI,  these  val¬ 
ues  were  -8.7  ±  13.8%,  -1.4  ±  20.2%,  and  -2.1  ±  17.7%. 

It  is  also  worth  noting  that  Vdiff  (see  Eq.  (9))  measures  the  pros¬ 
tate  volume  ratios  for  B-mode/MRI  and  VE/MRI  (Vdiff  =  VB.mode/ 
VMri  -  1).  Smith  et  al.  (2007)  report  a  MR/US  prostate  volume  ratio 
of  1.11  ±0.10  averaged  over  prostate  surfaces  from  10  patients 
outlined  by  seven  observers,  repeated  twice.  Similar  results  from 
other  work  have  been  listed  in  a  table  therein.  The  MR/US  prostate 
volume  ratios  for  the  nine  patients  in  our  database  are  1.12  ±  0.18, 
1.05  ±  0.21,  and  1.05  ±  0.19,  for  the  three  observers,  which  support 
previous  work.  For  MR/VE,  these  values  are  1.07  ±  0.13, 1.10  ±  0.10, 
and  1.02  ±  0.13.  However,  the  volume  ratio  or  Vdiff  only  report  dif¬ 
ferences  in  the  size  of  the  prostate,  whereas  two  prostate  surfaces 
can  be  greatly  different  in  shape  but  have  equal  volumes.  Verr, 
which  we  have  included  in  this  paper,  can  provide  useful  charac¬ 
terization  of  the  shape  difference. 


Currently,  one  dimensional  axial  motion  estimation  is  used  for 
creating  the  VE  images.  Any  out  of  plane  motion,  including  the  lat¬ 
eral  motions  caused  by  the  slippage  between  the  protective  sheath 
on  the  probe  and  the  surface  of  the  rectum  during  VE  data  collec¬ 
tion,  can  affect  the  VE  images.  This  can  be  improved  upon  by  using 
2D  motion  estimation  techniques  to  account  for  lateral  tissue  mo¬ 
tion.  We  have  not  used  2D  motion  tracking  because  most  of  the 
displacement  estimates  between  consecutive  frames  of  RF  data 
are  sub-sample.  We  have  shown  that  the  gain  based  from  2D  mo¬ 
tion  estimation  in  such  cases  is  not  significant  enough  to  warrant 
its  use  (Zahiri-Azar  et  al.,  2010)  unless  beam-steering  is  employed, 
such  that  the  lateral  motion  is  estimated  from  two  axial  measure¬ 
ments  at  different  beam  angles,  e.g.  ±10°.  The  increase  in  the  num¬ 
ber  of  axial  measurements  lowers  the  sampling  rate  and  we  opted 
not  to  do  that  in  this  set  of  experiments. 

One  should  also  keep  in  mind  that  the  presence  of  the  TRUS 
probe  results  in  deformation  of  the  posterior  region  of  the  gland 
in  both  VE  and  B-mode  surfaces.  For  more  accurate  registration 
of  these  surfaces  with  those  from  MRI,  this  deformation  could  be 
accounted  for.  However,  since  the  B-mode  and  VE  images  that 
we  use  in  this  paper  are  exactly  registered  to  each  other,  this  defor¬ 
mation  will  only  result  in  a  bias  in  the  volume  error  and  volume 
difference,  especially  in  the  posterior  region,  with  MRI  surfaces. 
The  result  of  such  a  bias  can  be  seen  to  be  a  generally  larger  mean 
and  standard  deviation  in  the  volume  error  and  volume  difference 
of  the  posterior  region  in  all  three  observers,  compared  to  that  of 
the  lateral  and  anterior  regions. 

It  is  important  to  note  that  several  papers  have  reported  in  vivo 
and  in  vitro  studies  on  healthy  vs.  cancerous  human  prostate  tissue 
showing  that  prostate  tumors  are  stiffer  than  normal  tissue  (Parker 
et  al.,  2011;  Zhang  et  al.,  2008;  Kamoi  et  al.,  2008;  Salomon  et  al., 
2008).  In  a  study  by  Kallel  et  al.  (1999)  elastography  images 
showed  contrast  between  anatomical  structures  of  freshly  excised 
canine  prostates.  The  central  gland  was  consistently  softer  than  the 
peripheral  gland.  In  the  current  work  we  addressed  in  vivo  valida¬ 
tion  of  ultrasound  elastography  as  a  tool  for  delineation  of  the  hu¬ 
man  prostate  gland  itself. 

As  a  final  note,  the  best  validation  of  the  prostate  gland  segmen¬ 
tation  could  be  a  comparison  to  pathology,  the  undisputed  gold 
standard.  Such  a  study  requires  access  to  the  whole  mount  pathol¬ 
ogy,  and  viable  solutions  to  the  open  problem  of  how  to  register 
the  shrunk,  misaligned  pathology  slices  to  pre-operative  images. 

We  have  quantitatively  shown  that  VE  is  a  promising  imaging 
modality  for  delineation  of  the  prostatic  region  and  the  use  of  such 
data  along  with  B-mode  ultrasound  can  improve  the  visualization 
of  the  gland.  Determination  of  the  clinical  value  of  VE  in  cancer 
detection  is  the  subject  of  our  future  research. 
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Fig  1.  Sample  T2  (left)  and  ADC  (right)  images  for  target  selection. 


Fig  2.  The  distribution  of  Ktrans  and  ADC  in  cancer  and  normal  ROIs. 


Introduction  -  With  an  estimated  240,890  cases  to  be  diagnosed  in  2011,  prostate  cancer  (PCa)  remains  among  the  most  common  male  malignancies 
with  an  annual  death  toll  of  33,270  men.  Prostate  tumors  are  not  visible  in  the  transrectal  ultrasound  (TRUS)  B-mode  images  used  for  the  guidance  of 
routine  biopsies.  Therefore,  TRUS  biopsy  is  not  targeted.  There  is  strong  evidence  that  multiparametric  MRI  (mpMRI)  including  Diffusion  Weighted 
(DW)  and  Dynamic  Contrast-Enhanced  imaging  (DCE)  in  addition  to  the  T2-weighted  MRI  can  improve  the  accuracy  of  PCa  localization  and 
staging  [1].  We  have  reported  transperineal  MR-guided  prostate  biopsy  with  highly  accurate  targeting  as  a  solution  to  improve  the  accuracy  of  the 
biopsy  process  in  detection  of  prostate  cancer  [2,  3].  The  targets  are  selected  by  a  team  of  three  radiologists,  based  on  pre-operative  mpMRI  (Fig.  1). 
The  objective  of  the  work  presented  here  is  to  assess  the  feasibility  of  quantitative  analysis  of  mpMRI  aided  by  a  machine  learning  approach  to  assist 
the  target  selection  process.  The  idea  is  to  train  a  classifier  on  the  mpMRI  data  from  radical  prostatectomy  patients  with  histologically  confirmed 
PCa.  The  resulting  classifier  is  used  retrospectively  on  the  data  from  patients  that  have  undergone  MR-guided  prostate  biopsy,  in  the  areas  selected  as 
targets  by  the  radiologists,  to  compute  a  measure  of  the  probability  of  cancer  and  compare  with  the  histological  analysis  of  the  core  biopsy  samples. 

Materials  and  Methods  -  Data:  All  patients  were  enrolled  in  a 
prospective  clinical  study  approved  by  the  institutional  review  board.  All 
patients  underwent  mpMRI  exams  in  a  GE  3T  MR  scanner  using  an 
endorectal  coil  and  included  Tlw,  T2w,  DWI  and  DCE  sequences.  The 
training  group  (N=13)  comprised  of  the  patients  that  had  confirmed  PCa  and 
underwent  radical  prostatectomy.  In  this  group,  an  expert  radiologist 
contoured  areas  of  the  prostate  peripheral  zone  (PZ)  that  were  cancerous  as 
reported  in  the  histopathology  report  and  suspicious  on  T2W  (low  SI),  raw 
DCE  (rapid  enhancement  and  wash  out  after  gadolinium  agent 
administration)  and  Average  Diffusion  Coefficient  ADC  maps  (low  SI) 
sequences.  The  opposite  characteristics  were  considered  true  of  normal 
prostate  PZ.  This  yielded  16  cancerous  and  5  normal  regions  with  a  total 
voxel  count  of  1499.  The  second  group  of  patients  included  all  of  the  cases 
that  have  undergone  3T  MR-guided  biopsy  with  our  current  protocol  in 
place  as  of  January  2011,  and  have  at  least  one  biopsy  target  in  the  PZ. 

This  amounted  to  five  cases  with  a  total  of  ten  PZ  biopsies. 

Classification  approach:  The  values  of  ADC  from  DWI  b0-500, 
and  Ktran\  ve,  time  to  peak  (TTP)  and  area  under  the  curve  (AUC)  from 
perfusion  imaging  were  available  for  each  pixel.  It  was  observed  that  only 
the  values  of  ADC  formed  distinctly  different  distributions  in  the  cancer  vs. 
normal  regions  in  the  training  dataset  (Figure  2).  We  employed  two  machine 
learning  approaches.  First  we  used  our  previously  reported  multi-feature 

support  vector  machine  (SVM)  for  mpMRI  classification  [4]  with  all  possible  combinations  of  the  five 
available  parameters.  Training  and  cross-validation  was  performed  on  the  prostatectomy  dataset  with  a 
leave-one -patient-out  validation  scheme.  The  best  outcome  was  achieved  when  only  ADC  was  used  as  the 
sole  feature.  Guided  by  this  outcome,  we  then  used  a  single-feature  Bayesian  framework  using  only  ADC 
to  build  a  machine  learning  solution  to  predict  the  outcome  of  the  biopsies.  In  the  Bayesian  approach, 
based  on  the  bell-shaped  distributions  of  the  ADC  values  in  the  two  classes,  we  assumed  Gaussian 
distributions  for  the  likelihood  of  ADC  values  in  cancer  P(ADC=a  \  C)  and  normal  areas  P(ADC=a  \  N). 

The  mean  and  standard  deviations  for  these  distributions  were  derived  from  a  Gaussian  fit  to  the  ADC  in 
each  class.  The  priors  for  cancer  and  normal  were  set  to  P(C)  =  P(N)  =  0.5.  Using  the  Bayes  rule,  we 
computed  the  posterior  probability  of  cancer  Pc  =  P(C\a)  =  P(a\C)P(C)  / [P(a\C)P(C)+P(a\N)P(N)]. 

Results  -  The  leave-one -patient-out  cross  validation  in  the  prostatectomy  dataset  yielded  an 
area  under  the  ROC  curve  of  0.966  using  SVM  with  ADC  as  the  only  feature.  It  was  noted  that  adding  the 
other  features  to  ADC  did  not  improve  this  results.  In  a  similar  cross-validation,  the  Bayesian  classifier 
resulted  in  an  area  under  ROC  curve  of  0.964.  Then,  using  the  Bayesian  classifier,  with  the  likelihoods 
estimated  on  all  the  13  training  cases,  we  created  maps  of  Pc  values  for  the  patients  in  the  second  group. 

The  results  are  listed  in  Table  1  and  show  that  in  eight  of  the  ten  PZ  targets,  the  classification  outcome 
was  confirmed  by  the  biopsy  finding  (Pc  close  to  1  for  cancer  and  Pc  <  0.5  for  benign  findings).  All  three 

cancer  outcomes  are  correctly  classified.  Five  of  the  seven  benign  outcomes  are  also  correctly  identified.  This  outcome  suggests  that  the  classification 
method,  if  validated  on  a  larger  sample  size,  has  the  potential  to  be  used  to  improve  the  biopsy  yield  by  testing  the  regions  of  the  proposed  targets, 
maintaining  a  high  sensitivity  and  removing  some  of  the  false  suggested  targets. 

Conclusions  and  limitations  -  In  this  retrospective  feasibility  study,  we  conducted  a  preliminary  evaluation  of  a  classifier  in  identifying 
cancer  areas  in  the  prostate  peripheral  zone.  Our  results  show  that  classification  results  in  the  majority  of  cases  agree  with  the  histopathology  analysis 
of  the  collected  sample.  The  process  of  ROI  selection  for  training  in  this  work  was  guided  by  approximate  histology  location,  the  ADC  maps  and  raw 
DCE  data.  Among  the  five  parameters  we  considered,  only  ADC  was  used  directly  for  visual  differentiation  in  the  training  stage  which  likely 
explains  the  strong  separation  in  distributions  we  observed.  Our  results  confirm  the  value  of  ADC  in  PCa  characterization  and  suggest  that  the 
supervised  classification  approach  may  have  assistive  value  in  target  selection  for  biopsy  planning.  In  the  future,  we  will  move  towards  using 
wholemount  histopathology  as  the  gold  standard  for  ROI  selection  in  the  training  stage.  This  might  also  enable  us  to  extend  the  methodology  beyond 
the  peripheral  zone.  The  number  of  patients  in  both  training  and  testing  stages  will  increase  as  we  continue  our  3T  MR  guided  biopsy  program. 
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Pc 

PI  -RPZ 

Benign 

0.0 

Pl-LPZ 

Benign 

0.4 

Pl-LPZ 

Benign 

0.9 

P2  -  LPZ 

Benign 

0.4 

P2  -  LPZ 

Benign 

0.0 

P3  -  Mid  RPZ 

3+4 

1.0 

P4  -  Mid  LPZ 

Benign 

0.9 

P5-  RPZ  (Apex) 

3+3 

0.9 

P5-  RPZ  (Base) 

3+3 

0.8 

P5  -  LPZ 

Benign 

0.1 

Table  1.  The  biopsy  findings  and  the 
computed  value  of  Pc. 
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Abstract.  Intraoperative  dosimetry  during  prostate  brachytherapy  is 
a  long  standing  clinical  problem.  We  propose  a  novel  framework  to  ad¬ 
dress  this  problem  by  reliable  detection  of  a  subset  of  seeds  from  3D 
transrectal  ultrasound  and  registration  to  fluoroscopy.  Seed  detection  in 
ultrasound  is  achieved  through  template  matching  in  the  RF  ultrasound 
domain  followed  by  thresholding  and  spatial  filtering  based  on  the  fixed 
distance  between  stranded  seeds.  This  subset  of  seeds  is  registered  to  the 
complete  reconstruction  of  the  implant  in  C-arm  fluoroscopy.  We  report 
results,  validated  with  a  leave-one-needle-out  approach,  both  in  a  phan¬ 
tom  (average  post-registration  seed  distance  of  2.5  mm)  and  in  three 
clinical  patient  datasets  (average  error:  3.9  mm  over  113  seeds). 


1  Introduction 

Low  dose  rate  brachytherapy  is  a  minimally  invasive  therapeutic  procedure  for 
prostate  cancer  that  has  rapidly  gained  acceptance  due  to  highly  successful  clini¬ 
cal  results.  In  this  procedure,  a  number  of  small  radioactive  sources  or  seeds  (125I 
or  103 Pd)  are  permanently  implanted  into  the  prostate  using  brachytherapy  nee¬ 
dles.  The  aim  is  to  deliver  a  sufficient  radiation  dose  to  kill  cancerous  tissue 
while  limiting  the  dose  in  radio-sensitive  regions  such  as  the  bladder,  urethra 
and  rectum.  Transrectal  ultrasound  (TRUS)  is  used  to  intraoperatively  guide 
the  transperineal  insertion  of  needles.  As  a  result  of  prostate  edema,  motion  of 
the  gland  due  to  needle  forces,  and  possible  intra-operative  changes  to  the  plan 
due  to  various  factors  such  as  interference  with  the  pubic  arch,  the  locations  of 
the  implanted  seeds  do  not  necessarily  match  with  the  initial  treatment  plan. 
Hence,  for  quality  assurance,  intra-operative  dosimetry  is  highly  beneficial. 
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Researchers  have  approached  ultrasound-based  seed  detection  [1].  However, 
accurate  seed  localization  based  on  ultrasound  has  proven  to  be  a  very  difficult 
task  due  to  clutter  from  other  highly  reflecting  objects  such  as  calcifications  re¬ 
sulting  in  false  positive  appearances,  seed  specularity  and  shadowing,  and  limited 
field  of  view.  Even  when  hand-segmented,  up  to  25%  of  the  seeds  remain  hid¬ 
den  in  ultrasound  images  [2].  Therefore,  C-arm  fluoroscopy  is  commonly  used 
for  visual  assessment  of  the  implanted  seeds.  However,  the  prostate  gland  is 
not  discernible  in  fluoroscopy  images.  Fusion  of  the  fluoroscopy  images  and  ul¬ 
trasound  is  therefore  considered  as  a  possible  solution  [3-6].  If  complete  seed 
localization  and  implant  reconstruction  from  fluoroscopy  images  is  available,  the 
registration  of  the  result  to  ultrasound  will  enable  dosimetry.  In  recent  years, 
the  fluoroscopy  reconstruction  problem  has  been  extensively  addressed.  Given 
3-5  fluoroscopy  images,  and  the  relative  pose  of  the  C-arm  in  each  acquisition, 
the  back-projection  technique  can  be  used  to  reconstruct  the  3D  implant  [7]. 
For  registration,  attaching  fiducial  markers  to  the  ultrasound  probe  [8],  using 
the  ultrasound  probe  itself  as  a  fiducial  [4],  or  using  the  seeds  as  fiducials  [9] 
have  been  mentioned.  Due  to  patient  and  equipment  motion  between  the  acqui¬ 
sition  of  ultrasound  and  fluoroscopy,  registration  based  on  static  markers  is  not 
reliable.  Furthermore,  the  use  of  fiducial  markers  is  an  unwelcome  addition  to 
the  ordinary  setup  in  the  operating  room  due  to  time  and  space  limitations. 

In  this  work  we  propose  using  a  subset  of  seeds  extracted  from  ultrasound 
images  to  perform  point-based  registration  between  the  seed  clouds  from  fluo¬ 
roscopy  and  3D  ultrasound.  We  present  several  technical  innovations.  Instead  of 
conventional  B-mode  ultrasound,  we  use  RF  signals  processed  to  enhance  seed 
contrast.  Template  matching  with  a  variety  of  in  vivo  and  ex  vivo  seed  tem¬ 
plates  is  reported.  To  enable  dosimetry,  we  have  devised  a  two  stage  strategy 
consisting  of  first  2D  registration  of  needle  projections  from  the  ultrasound  and 
fluoroscopy,  followed  by  the  3D  registration  of  only  the  seeds  in  the  matched 
needles.  We  provide  the  results  of  this  approach  on  both  clinical  and  phantom 
data.  This  novel  methodology  targets  a  complicated  and  long  standing  problem, 
with  no  addition  to  the  routine  therapeutic  procedure.  We  show  that  our  method 
has  good  promise  to  address  this  clinical  challenge. 


2  Methods 

The  outline  of  our  methodology  is  presented  in  Figure  1.  For  reconstruction  of 
the  implant  in  fluoroscopy,  we  implemented  and  used  the  method  described  in 
[7].  The  steps  for  acquiring  the  fluoroscopy  reconstruction  included  C-arm  pose 
estimation  from  rotation  angle  and  compensation  for  sagging,  followed  by  back- 
projection  of  the  seeds,  and  finally  seed  to  needle  assignment  using  a  minimum 
cost  flow  (MCF)  algorithm.  The  outcome  was  validated  in  terms  of  number  of 
seeds,  needles,  and  seed  to  needle  assignments  based  on  the  brachytherapy  plans, 
both  in  patient  and  phantom  datasets.  In  this  article  we  focus  on  ultrasound- 
based  partial  seed  detection,  needle  matching  and  registration  from  ultrasound 
to  fluoroscopy. 
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2.1  Seed  Detection 

3D  Ultrasound  Setup  and  Data:  We  developed  a  3D  ultrasound  system 
based  on  a  brachytherapy  stepper  (EXII,  CIVCO  Medical  Solutions)  modi¬ 
fied  by  motorizing  the  cradle  rotation.  The  sagittal  array  of  a  dual  plane  lin¬ 
ear /micro  convex  broadband  5  —  9  MHz  endorectal  transducer  (Ultrasonix)  was 
used.  RF  data  was  recorded  at  a  frame  rate  of  42  fps,  during  the  probe  rotation 
from  -45°  to  50°  (0°  corresponded  to  the  probe  aligned  with  the  central  sagittal 
cross  section  of  the  prostate  gland).  2D  frame  size  was  5x5.5  cm.  We  present 
the  results  of  our  work  on  data  collected  immediately  after  seed  implantation 
in  the  OR,  from  three  brachytherapy  patients  in  Vancouver  Cancer  Center.  We 
also  present  data  from  a  CIRS  Model  053  tissue-equivalent  ultrasound  prostate 
phantom  (CIRS,  Inc.,  Norfolk,  VA).  For  this  phantom,  a  plan  consisting  of  135 
seeds  and  26  needles  was  created  which  was  carried  out  by  a  radiation  oncologist. 


RF  Signal  Processing:  In  order  to  improve  the  seed  to  background  contrast, 
we  averaged  the  signal  power  over  windowed  blocks  of  the  RF  signals.  In  other 
words,  we  replaced  a  segment  of  length  n  at  depth  d  of  an  RF  line  with  the 
reflected  power  Pd  computed  as: 
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Fig.  1.  Workflow  of  the  seed  detection  method 
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pu  ELi^fcMfc)2  (1) 

n 

where  x(k)  (k  =  l,...,n)  are  the  samples  in  the  RF  segment,  and  w(k)  are 
Hamming  window  weights.  This  step  was  applied  with  n  =10  and  a  50%  over¬ 
lap  between  consecutive  blocks.  We  have  previously  reported  that  this  process 
doubles  the  contrast  to  noise  ratio  (CNR)  between  the  seed  regions  and  the 
background  [10].  Furthermore,  the  resulting  five  fold  reducion  in  the  size  of  the 
RF  signals  improved  the  speed  of  the  template  matching  step. 

Seed  Templates:  Simple  thresholding  of  the  contrast  enhanced  RF  data  results 
in  a  large  number  of  seed  candidates  and  seed-like  clutter.  In  order  to  reduce  false 
positive  detections,  we  computed  the  normalized  cross  correlation  of  the  seed 
regions  with  seed  templates.  We  experimented  with  three  groups  of  templates 
and  will  provide  a  comparison. 

Ex  vivo  templates:  We  created  a  3D  template  acquired  by  imaging  a  seed  in 
water,  placed  parallel  to  the  probe  at  the  center  of  the  ultrasound  probe.  The 
3D  template  was  formed  by  rotating  the  probe  and  acquiring  21  sagittal  images. 

In  vivo  templates:  In  the  clinical  situation,  the  existence  of  background  tis¬ 
sue,  blood  and  edema  significantly  alters  the  appearance  of  seeds  on  ultrasound 
images  compared  to  the  described  ex  vivo  templates,  resulting  in  low  normal¬ 
ized  cross  correlation  values.  Therefore,  we  created  a  second  set  of  templates 
extracted  from  in  vivo  data.  These  were  extracted  from  different  cases  to  ensure 
that  the  template  extracted  from  a  specific  patient  dataset,  was  not  used  for  seed 
detection  in  that  dataset.  They  were  acquired  by  manually  clicking  the  center 
of  a  visually  distinct  seed  in  in  vivo  3D  data. 

In  vivo  and  ex  vivo  two-seed  templates:  We  also  created  templates,  both  in 
vivo  and  ex  vivo ,  in  which  the  template  area  contained  two  seeds.  The  two- 
seed  templates  were  examined  as  a  potential  solution  to  reduce  the  number  of 
false  positives,  given  the  fact  that  in  stranded  brachytherapy  performed  in  our 
institution,  the  distance  between  seeds  is  fixed  at  1  cm  with  very  few  excep¬ 
tions.  Therefore,  the  existence  of  two  strong  seed  candidates,  1  cm  apart,  is  an 
indication  of  a  true  detection. 

NCC-Based  3D  Template  Matching:  The  normalized  cross  correlation  of 
the  3D  template  g  and  a  cropped  area  of  the  image  /  equal  in  size  with  g  and 
centered  at  location  (i,j,k)  can  be  computed  as: 


,  k)  = 


(Ex^  i /. z)  ~  k^kUgjx,  y,  z)  -  a) 

y/[J2x,y,z(fi,jAx,y’z)  ~  fi,j,k)2J2XtytZ(9(x,y,x)  -s)2 


(2) 


where  x,  y  and  2  represent  the  directions  in  the  image  coordinate  system.  Com¬ 
putation  of  f 'kg  results  in  a  new  image  of  the  same  size  as  /,  with  values  in  range 
of  [0,1]  with  largest  values  representing  the  centers  of  areas  most  similar  to  the 
template.  The  frequency  domain  implementation  of  NCC  was  completed  in  under 
six  minutes  on  a  regular  PC  with  MATLAB  for  template  size  of  30x60x21,  and 
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image  size  of  128x258x391.  Note  that  NCC  is  not  invariant  to  scale.  However, 
in  our  case,  the  images  and  the  templates  were  acquired  with  similar  imaging 
parameters,  therefore  the  scales  matched. 

Thresholding,  Spatial  Filtering,  and  Grouping  in  the  NCC  Image:  The 

NCC  image  was  thresholded.  Starting  from  the  point  with  the  highest  NCC,  a 
neighborhood  of  the  size  of  the  seed  was  cleared  around  each  non-zero  voxel. 
This  was  necessary  because  each  seed  consists  of  several  bright  voxels,  while  we 
need  a  single  voxel  to  represent  the  seed.  The  remaining  voxels  were  grouped 
into  needles  using  the  Hough  transform  [11]  followed  by  eliminating  single  seeds 
that  could  not  be  grouped  into  lines.  Using  the  knowledge  of  the  fixed  1  cm 
distance  between  the  seeds  in  our  data,  we  applied  an  additional  trimming  step. 
On  each  needle,  starting  from  the  seed  with  the  highest  NCC  value,  any  other 
seed  candidate  that  was  within  0.8  cm  was  removed. 

2.2  Matching  and  Registration 

After  applying  a  transformation  that  matched  the  centers  of  mass  of  the  two 
datasets,  we  applied  a  2D  needle  matching  process.  This  provided  an  initial 
alignment  and  reduced  the  risk  of  local  minima  due  to  the  unbalanced  number 
of  seeds  in  ultrasound  and  fluoroscopy.  Matching  was  performed  by  applying  a  2D 
rigid  registration  between  the  needle  projections  on  the  transverse  plane  passing 
through  the  prostate  mid-gland  in  the  fluoroscopy  implant.  Assuming  that  X  is 
the  set  of  n  projection  points  from  ultrasound,  and  Y  is  the  set  of  m  projection 
points  from  fluoroscopy,  the  rotation  and  translation  parameters  of  the  transfor¬ 
mation  T  were  found  to  minimize  JA=1.n  dc[T(Xi),  Y]  where  dc\T{Xi),  Y]  is  the 
Euclidean  distance  of  the  ultrasound  projection  point  X{  from  its  closest  match 
in  the  point  set  Y.  After  the  matching  step,  the  fluoroscopy  needles  without  a 
match  in  ultrasound  were  removed  and  the  standard  3D  point-based  Iterative 
Closest  Point  (ICP)  registration  algorithm  [12]  was  applied. 

3  Results  and  Conclusions 

We  quantified  the  outcome  of  our  ICP  seed  registration  method  based  on  1) 
the  post-registration  distances  between  ultrasound  seeds  and  their  closest  fluo¬ 
roscopy  counterparts,  2)  the  stability  of  needle  matches  and  the  recorded  regis¬ 
tration  errors  subject  to  the  removal  of  any  of  the  detected  needles. 

Registration  Errors:  Table  1  presents  the  results  of  ultrasound  seed  detection 
and  registration  for  the  CIRS  phantom.  The  best  outcome  was  achieved  when 
the  two-seed  ex  vivo  template  set  was  used.  The  post-registration  seed  distances 
from  ultrasound  to  the  closest  matching  fluoroscopy  seed  was  2.48  mm.  Note 
that  the  best  registration  result  was  obtained  when  the  lowest  number  of  seeds 
(73  out  of  135,  17  needles  out  of  26)  were  detected.  The  result  of  the  2D  matching 
and  3D  registration  of  the  seed  clouds  for  the  phantom  data,  using  the  two-seed 
ex  vivo  template,  is  depicted  in  Figure  2. 
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Fig.  2.  Phantom  results:  2D  matching  of  needle  projections  in  transverse  plane  (left), 
3D  seed  cloud  registration  using  the  matched  needles  from  fluoroscopy  and  ultrasound 


Table  1.  Phantom  result:  #  of  detected  seeds,  registration  errors  with  all  detected 
needles,  and  average  registration  error  for  the  leave-one-needle-out  validation  test 


Phantom 

dist.  er.(mm) 
all  needles 

dist.  er. 
(mm) 
L.0.0 

#a 

seeds 

Total 

#  fl  seeds, 
in  matching 
needles 

#  of  detected 
seeds  in 
ultrasound 

1-seed,  ex  vivo  tempi. 

2.69±2.03 

2.71F1.98 

135 

95 

86 

1-seed,  in  vivo  tempi. 

4.33±2.21 

4.40±2.24 

135 

101 

95 

2-seed,  ex  vivo  tempi. 

2.48=bl.52 

2.48=bl.41 

135 

82 

73 

Table  2  presents  the  results  for  the  three  patient  datasets.  The  one- seed  and 
the  two-seed  templates  did  not  result  in  significantly  different  error  values.  We 
obtained  errors  of  3.36  mm,  3.73  mm,  and  4.76  mm  for  the  three  cases  on  in 
vivo  templates.  With  ex  vivo  templates,  we  witnessed  a  decrease  in  the  number 
of  detected  seeds  for  all  cases  and  an  error  increase  in  two  patients  with  a  slight 
improvement  in  case  3  in  terms  of  registration  error  (to  4.38,  4.08  and  4.22  mm). 
For  case  2,  Figure  3  illustrates  the  results  of  2D  matching  and  3D  registration. 
In  both  Figures  2  and  3  one  can  see  that  the  unmatched  needles  tend  to  be  from 
the  anterior  side  (top  of  the  images),  while  the  posterior  seeds  that  are  closest 
to  the  probe  are  accurately  detected.  This  is  likely  due  to  signal  attenuation  and 
the  depth  dependent  reduction  in  the  resolution  of  our  3D  ultrasound  system. 

Leave-one-needle-out  Validation  of  the  Registration  Process:  In  order 
to  study  the  stability  of  our  matching  and  registration  procedure,  we  also  ran 
a  leave-one- needle-out  experiment.  For  each  patient  and  the  phantom  case,  as¬ 
suming  that  n  ultrasound  needles  were  identified,  we  repeated  the  matching  and 
registration  procedure  n  times,  each  time  with  n  —  1  needles.  This  amounted  to 
the  removal  of  three  to  seven  (10%  to  20%)  of  the  seeds  in  each  round  for  patient 
cases.  The  idea  is  that  if  the  registration  is  valid,  and  not  just  a  local  minimum, 
the  removal  of  any  specific  needle  should  not  drastically  change  the  outcome. 

The  average  of  the  errors  in  the  leave  one  out  experiments  (column  3  in 
Tables  1  and  2)  were  close  to  the  errors  when  all  detected  seeds  were  used  in 
the  registration  step  (column  2).  We  also  examined  the  stability  of  the  needle 
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Fig.  3.  A  patient  case:  2D  matching  (left),  registered  seeds  in  matched  needles  (right) 


Table  2.  Number  of  the  detected  seeds,  registration  errors  and  leave- one- needle- out 
errors  for  the  3  patient  datasets 


Case 

dist.  er.(mm) 
all  needles 

dist.  er. 
(mm) 
L.0.0 

#a 

seeds 

Total 

#  H  seeds, 
in  matching 
needles 

#  of  detected 
seeds  in 
ultrasound 

PI  1-seed  in  vivo  tempi. 

3.72=41.86 

3.85=41.96 

102 

51 

35 

PI  2-seed  in  vivo  tempi. 

3.36=41.78 

3.44=41.95 

102 

57 

37 

PI  ex  vivo  tempi. 

4.07=42.10 

4.38=41.87 

102 

30 

16 

P2  1-seed  in  vivo  tempi. 

3.98=42.50 

3.77=42.27 

115 

74 

56 

P2  2-seed  in  vivo  tempi. 

3.73=41.86 

3.79=41.94 

115 

71 

49 

P2  ex  vivo  tempi. 

3.74=42.8 

4.08=42.58 

115 

55 

37 

P3  1-seed  in  vivo  tempi. 

4.76=41.87 

4.90=42.12 

100 

35 

27 

P3  2-seed  in  vivo  tempi. 

4.88=41.95 

4.79=42.25 

100 

33 

23 

P3  ex  vivo  tempi. 

4.524=1.65 

4.22=41.49 

100 

26 

16 

matching  step  subject  to  removal  of  needles.  In  the  case  of  the  phantom  data, 
and  case  1  among  the  patient  datasets,  it  was  noted  that  regardless  of  which 
needle  was  removed  in  the  leave  one  out  experiment,  the  matched  fluoroscopy 
needle  for  the  rest  of  ultrasound  needles  remained  the  same  (number  of  detected 
needles:  n=  17).  In  patient  cases  2  and  3,  on  average,  the  match  for  one  needle 
changed  due  to  the  removal  of  a  needle  (n  =14  and  n  =12). 


4  Conclusions 

We  showed  that  it  is  feasible  to  use  contrast  enhanced  RF  ultrasound  data,  tem¬ 
plate  matching,  and  spatial  filtering  to  detect  a  reliable  subset  of  brachytherapy 
seeds  from  ultrasound  to  enable  registration  to  fluoroscopy.  Real-time  implemen¬ 
tation  requires  the  matching  process  to  be  computationally  improved  though 
parallelization  to  enhance  the  current  computation  time  of  around  six  minutes. 
More  robustness  analysis  and  additional  registration  approaches  will  be  imple¬ 
mented  in  future  work.  The  3D  ultrasound  system  used  in  this  work  acquired 
sagittal  images  while  rotating  radially.  This  results  in  a  significant  decrease  in 
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spatial  resolution  at  increasing  distances  from  the  probe.  The  use  of  a  com¬ 
prehensive  depth  dependent  set  of  templates  can  improve  our  results.  It  is  also 
possible  to  perform  the  ultrasound  seed  detection  after  only  a  part  of  the  im¬ 
plant,  for  example  the  top  row  of  needles,  is  completed.  This  will  reduce  the 
shadowing  effects.  Clinical  data  is  being  acquired  to  test  these  possibilities. 
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Introduction:  Prostate  cancer  is  the  most  common  noncutaneous  malignancy  among  men.  The  clinical  routine  for  diagnosis  is  biopsy  under 
ultrasound  guidance.  However,  as  a  result  of  the  multi-focal  nature  of  the  disease,  clinically  significant  cases  of  cancer  can  be  missed,  resulting  in 
repeated  biopsies.  We  have  previously  shown  that  Diffusion  Tensor  Imaging  (DTI)  and  Dynamic  Contrast  Enhanced  (DCE)  MRI  provide  a  high 
degree  of  sensitivity  in  detecting  prostate  cancer  [1,2].  In  this  study  we  present  a  framework  of  classification  using  multi-parametric  MRI  feature 
vectors  and  Support  Vector  Machines  (SVMs)  to  detect  prostate  cancer  and  to  create  cancer  probability  maps  that  highlight  areas  of  tissue  with  high 
risk  of  cancer.  The  correlation  of  this  probability  with  Gleason  Grade  is  also  studied. 

Data:  Twenty  nine  men  with  a  high  clinical  suspicion  for  prostate  adenocarcinoma  were  included  in  this  study.  Ah  MRI  exams  were  carried  out  on  a 
3T  Philips  Achieva  MRI  scanner.  Twelve  axial  slices  (4  mm,  no  gap)  across  the  prostate  gland  were  acquired  for  both  DTI  and  DCE  MRI  data  with 
FOV  of  24  cm.  The  DTI  data  were  processed  off-line  to  generate  maps  of  Apparent  Diffusion  Coefficient  (ADC)  and  Fractional  Anisotropy  (FA)  [2]. 
Three  DCE-based  pharmacokinetic  parameters,  namely  volume  transfer  constant  Ktrans,  fractional  volume  of  the  extra-vascular  extra-cellular  space  ve, 
and  fractional  plasma  volume  vp,  were  calculated  by  fitting  the  Gd  concentration  vs.  time  curves  to  the  extended  Kety  model  [3].  After  imaging,  from 
8  to  12  needle  biopsies  were  collected  from  each  subject  depending  on  the  size  of  the  prostate.  The  dataset  included  a  total  of  240  negative  biopsy 
cores  and  29  positive  biopsy  cores.  The  histology  was  interpreted  with  assignment  of  the  Gleason  score  by  a  number  of  different  anatomic 
pathologists  who  practice  general  and  subspecialty  uropathology 

Classification  and  mapping  method:  Each  biopsy  core  in  the  dataset  was  represented  by  a  feature  vector  consisting  of  the  average  values  of  the  five 
DCE-DTI  features  ( X=  [Ktrans,  ve,  vp,  ADC,  FA])  in  its  corresponding  area  in  the  MRI  data.  Following  [2],  for  negative  biopsies,  this  corresponding 
area  was  set  to  be  the  entire  biopsy  target  volume  (e.g.  mid-left-lateral),  while  for  positive  cores,  this  area  was  determined  from  a  combination  of 
manual  segmentation  and  thresholding  based  on  biopsy  (see  [2]  for  details).  SVM  classification  with  radial  basis  kernel  function  [4]  was  used  to 
separate  the  data  into  “normal”  and  “cancer”  classes  based  on  these  feature  vectors.  The  parameters  of  the  SVM  kernel  function  were  chosen  in  a 
grid-based  search  [5,6].  The  training-testing  validation  was  performed  in  leave-one -patient-out  manner.  For  each  patient,  we  trained  the  SVM  based 
on  biopsy  cores  from  all  other  patients  and  tested  it  on  the  cores  from  the  patient  in  question.  To  acquire  post-classification  cancer  probabilities  (Pc  = 
P(cancer\X)) ,  the  extension  of  the  SVM  training  proposed  by  Platt  [7]  was  used.  The  /Rvalue  acquired  for  each  biopsy  core  was  used  as  a  decision 
threshold  for  cancer  detection.  The  Receiver  Operating  Characteristic  (ROC)  curve  was  obtained  by  setting  this  detection  threshold  to  values  in  the 
range  of  [0,1].  Given  A  for  all  pixels  in  one  MRI  slice,  cancer  probability  maps  were  created  by  calculating  and  plotting  the  values  of  Pc  -  using  the 
SVM  trained  on  data  from  all  other  subjects  -  for  the  entire  prostate  region  in  the  slice. 

Results  and  discussion:  The  ROC  curves  were  obtained  separately  for  DCE  features  XDCE  =  [Ktrans,  ve,  vp],  for  DTI  features  XDTI  =  [ADC,  FA],  and 
for  the  combined  feature  vector  X=  [Ktrans,  ve,  vp,  ADC,  FA]  resulting  in  area  under  ROC  curve  (AUC)  values  of  0.867,  0.919  and  0.956  respectively 
(Fig.l  ).  The  combined  feature  vector  resulted  in  higher  AUC  than  DCE  (p=0.002)  and  DTI  (p=0.01).  With  the  combined  feature  vector,  at  the 
decision  threshold  of  Pc  =  0.5,  three  of  the  29  tumors  were  misclassified  while  a  specificity  of  91%  was  obtained.  At  the  decision  threshold  ofPc  = 
0.7,  only  one  tumor  was  misclassified,  while  a  high  specificity  of  87%  was  maintained.  We  also  noted  a  correlation  between  the  /Rvalue  and  the 
Gleason  grade  of  the  tumors.  The  average  Pc  value  was  0.555  for  tumors  of  grade  3+3  (number  of  tumors=7),  0.778  for  tumors  of  grade  3+4  and  4+3 
(n=19),  and  0.963  for  grade  4+5  (n  =  3).  The  increase  in  Rvalues  was  significant  from  Grade  6  to  7  (p=0.01).  The  small  number  of  tumors  in  grade 
4+5  did  not  warrant  an  analysis  of  statistical  significance.  None  of  the  4+5  tumors  were  misclassified  in  any  of  the  experiments.  The  T2W  image  and 
the  cancer  probability  map  obtained  for  the  mid  slice  MRI  data  of  a  patient  with  positive  biopsy  cores  in  the  mid-left-lateral  and  mid-left-medial 
regions  of  the  prostate  are  illustrated  in  Fig.  2. 

Conclusions:  Our  result  is  in  agreement  with  the  previous  work  that  shows  the  diagnostic  power  of  combined  DTI  and  DCE  MRI  [2].  The  developed 
SVM-based  probabilistic  approach  to  cancer  detection  provides  a  high  level  of  sensitivity  while  maintaining  high  specificity.  The  observed  increase 
in  cancer  probability  with  increasing  Gleason  grade  is  promising.  This  suggests  that  multi-class  SVM-based  classification  is  a  potential  tool  for  non- 
invasive  grading,  given  a  sufficiently  large  dataset  to  enable  training  on  tumors  of  different  grades. 
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receiver  operating  characteristic  (ROC) 


Figure  1-  ROC  curves,  for  different  groups  of  features 
acquired  by  changing  the  decision  threshold,  Pc,  from  0  to  1 . 


Figure  2  -  Left:  T2W  image  of  mid  region  of  a  patient  with  biopsy  confirmed 
cancer  in  mid-left  region.  Right:  The  SVM-based  cancer  probability  map, 
with  hot  colors  corresponding  to  higher  Pc.  The  tumor  is  distinguished. 
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Abstract.  In  this  paper  we  propose  a  fully  automatic  2D  prostate  seg¬ 
mentation  algorithm  using  fused  ultrasound  (US)  and  elastography  im¬ 
ages.  We  show  that  the  addition  of  information  from  mechanical  tissue 
properties  acquired  from  elastography  to  acoustic  information  from  B- 
mode  ultrasound,  can  improve  segmentation  results.  Gray  level  edge  sim¬ 
ilarity  and  edge  continuity  in  both  US  and  elastography  images  deform 
an  Active  Shape  Model.  Comparison  of  automatic  and  manual  contours 
on  107  transverse  images  of  the  prostate  show  a  mean  absolute  error  of 
2.6  ±0.9  mm  and  a  running  time  of  17.9  ±  12.2  s.  These  results  show  that 
the  combination  of  the  high  contrast  elastography  images  with  the  more 
detailed  but  low  contrast  US  images  can  lead  to  very  promising  results 
for  developing  an  automatic  3D  segmentation  algorithm. 


1  Introduction 

Low  dose  rate  (LDR)  prostate  brachytherapy  is  a  common  method  for  treating 
patients  with  low  risk  prostate  cancer.  In  this  treatment,  40-100  small  radioactive 
seeds  are  permanently  inserted  in  the  prostate  and  its  periphery.  Treatment 
planning  and  delivery  relies  on  transrectal  ultrasound  (TRUS)  imaging.  In  order 
to  create  the  treatment  plan,  a  pre-operative  volume  study  is  carried  out  in 
which  a  set  of  transverse  ultrasound  images  are  collected.  These  images  are  then 
manually  segmented  to  extract  the  prostate  boundaries.  A  plan  is  devised  to 
deliver  sufficiently  high  radiation  dose  to  the  cancerous  tissue  while  maintaining 
a  tolerable  dose  to  healthy  tissue.  Reliable  segmentation  and  visualization  of  the 
prostate  is  a  vital  step  in  dose  planning.  Manual  segmentation  is  time  consuming 
and,  due  to  the  low  signal  to  noise  ratio  of  ultrasound  images,  inter  and  intra¬ 
observer  variabilities  are  high.  Even  though  various  2D  prostate  segmentation 
methods  and  some  3D  methods  have  been  proposed  in  the  literature  [1,2, 3, 4, 5], 
the  effective  automatic  segmentation  of  ultrasound  images  of  the  prostate  has 
remaining  challenges  such  as  user  initialization  and  limited  accuracy. 

In  this  paper,  we  propose  a  prostate  segmentation  method  based  on  intra¬ 
modality  fusion  of  ultrasound  B-mode  and  elastography.  Elastography  [6],  in 
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which  mechanical  properties  of  tissue  are  characterized,  has  shown  to  be  promis¬ 
ing  in  improving  the  visibility  of  the  prostate  gland  [7,8].  In  a  recent  study,  we 
have  shown  that  ultrasound  dynamic  elastography  images  of  the  prostate  have 
superior  object-background  contrast  compared  to  B-mode  ultrasound,  especially 
at  the  base  and  apex  [9].  This  is  due  to  the  fact  that  prostate  tissue  is  gener¬ 
ally  stiffer  than  the  surrounding  tissue.  We  utilize  this  advantage  and  combine 
elastography  and  US  image  data  for  2D  segmentation  of  the  prostate.  The  elas¬ 
tography  images  are  acquired  using  a  system  described  in  [9]  which  enables  the 
simultaneous  registered  acquisition  of  B-mode  and  elastography  images,  thereby 
eliminating  the  concerns  about  image  registration  and  cost. 

We  use  an  Active  Shape  Model  (ASM)  [10]  approach  which  starts  with  an 
initial  shape  extracted  from  a  large  number  of  elastography  and  B-mode  prostate 
images.  The  deformation  of  this  initial  shape  is  restricted  to  conform  to  the 
statistical  model  and  is  guided  by  edge  detection  from  both  elastography  and 
B-mode  images  based  on  edge  gradient  similarity  and  continuity.  The  use  of  a 
statistically  created  model  ensures  the  compliance  of  the  resulting  contours  with 
the  overall  shape  of  the  organ.  Additionally,  the  restricted  deformation  results 
in  robustness  to  poor  image  quality.  The  use  of  a  measure  of  edge  continuity  [9] 
in  addition  to  gradient  similarity,  reduces  the  effects  of  strong  speckle-induced 
local  edges  on  the  algorithm  which  improves  the  rate  of  convergence.  We  provide 
a  statistical  analysis  of  the  accuracy  of  our  2D  image  segmentation  method  and 
show  that  the  combined  use  of  elastography  and  B-mode  images  improves  the 
accuracy  and  the  convergence  rate.  Further,  we  describe  a  preliminary  framework 
for  extending  the  proposed  method  to  an  automatic  3D  segmentation  algorithm 
and  present  an  example. 

2  Methods 

The  US  and  elastography  images  used  in  this  paper  were  acquired  from  pa¬ 
tients  going  through  the  standard  LDR  prostate  brachytherapy  procedure  at 
Vancouver  Cancer  Center,  BC  Cancer  Agency.  Intra-operatively,  prior  to  the 
procedure,  RF  data  and  US  images  were  simultaneously  collected  using  the  sys¬ 
tem  described  in  [9] .  In  this  system  RF  data  is  collected  at  approximately  40  fps 
from  the  sagittal  array  of  a  vibrating  (amplitude  0.5-2  mm,  frequency  range  2- 
10  Hz)  and  rotating  (—45°  to  50°)  TRUS  probe  (dual-plane  linear /micro convex 
broadband  5-9  MHz  endorectal  transducer,  Ultrasonix  Medical  Corp.).  The 
RF  data  were  processed  [11]  to  obtain  sagittal  elastography  strain  images  from 
which  conventional  transverse  images  of  the  prostate  were  achieved. 

Our  2D  segmentation  approach  combines  the  information  from  elastography 
and  US  images  within  an  Active  Shape  Model  (ASM)  which  deforms  based 
on  gray  level  similarity  and  edge  continuity.  We  follow  the  approach  from  [10]. 
First,  we  construct  a  training  set  by  manually  selecting  30  specific  points  on 
the  prostate  boundary  in  N  =  25  mid-gland  images  from  7  patients.  The  set 
of  images  used  for  creating  the  training  set  does  not  include  the  images  to  be 
segmented.  The  manually  segmented  contours  are  aligned  by  using  least-squares- 
based  iterative  scaling,  rotation  and  translation  and  the  average  of  the  resulting 
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contours,  x,  is  used  as  the  initial  contour.  Then,  in  order  to  capture  the  statistics 
of  the  training  set,  we  calculate  the  covariance  matrix,  S  =  J2iLi  dxidxf ,  where 
dxi  in  the  training  phase  is  the  distance  between  each  point  on  the  manual 
contour  and  the  corresponding  point  on  the  mean  shape.  The  modes  of  variation 
of  the  shape  are  described  by  the  eigenvalues,  A and  eigenvectors,  pi,  of  S', 
from  which  the  t  largest  eigenvalues  are  selected  as  the  most  significant  modes 
of  variation.  We  selected  t  such  that  for  i  =  1  ,•••,£,  A^/^A  >  5%.  A  shape 
instance  consistent  with  the  training  set  can  thus  be  created  using  Eq.(l): 


x  =  x  +  Pb  (1) 

where  P  =  (pi  •  •  •  Pt)  is  the  matrix  of  the  first  t  eigenvectors  and  b  =  (bi  •  •  •  bt)T  is 
a  vector  of  weights.  Hence,  in  the  shape  fitting  phase,  given  a  shape  deformation, 
dx,  the  shape  parameter  adjustments,  db,  can  be  calculated  using  Eq.(2). 


db  =  PTdx  (2) 

To  calculate  the  movement,  chq,  for  each  model  point  i,  a  measure  of  edge  gray 
level  similarity  was  used  in  [10].  For  each  point  i  of  each  image  j  of  the  training 
set,  a  normalized  edge  derivative  profile,  gij  normal  to  the  boundary,  centered  at 
the  model  point  and  of  length  np  is  extracted,  gij  is  averaged  over  all  images  from 
which  the  covariance  matrix  S9i  is  calculated  to  obtain  a  statistical  description 
of  the  gray  level  appearance  for  every  point.  During  the  shape  fitting,  at  each 
iteration  and  for  every  point,  sample  edge  derivative  profiles  hi(d)  of  distance 
d  (d  =  —/,•••,/)  from  the  boundary  point  and  length  np  are  extracted  in  a 
similar  manner.  The  square  of  the  Mahalanobis  distance  of  these  profiles  from 
the  model  profile,  give  a  measure  of  edge  gray  level  similarity.  For  each  point, 
the  d  resulting  in  the  least  Mahalanobis  distance  suggests  the  required  point 
movement  along  a  line  normal  to  the  boundary.  The  physical  values  of  l  and  np 
are  set  to  8  and  5  mm,  respectively. 

In  our  data  set,  we  observed  that  the  gray  level  edge  similarity  measure  alone, 
gives  many  false  positives  due  to  ultrasound  speckle  or  sharp  edge-like  structures 
since  only  ID  information  (normal  to  the  edge)  is  being  analyzed.  Therefore,  in 
our  approach,  we  incorporate  our  edge  continuity  measure  [9]  which  measures 
the  continuity  of  the  edge  in  a  direction  orthogonal  to  the  edge  profile.  At  a  dis¬ 
tance  d  from  each  point,  we  compute  the  average  normalized  cross-correlation, 
c*(d),  between  the  edge  intensity  profile  ei{d)  (obtained  similar  to  g^ ) ,  of  length 
rip,  with  its  two  neighboring  left  and  right  edge  intensity  profiles.  For  a  contin¬ 
uous  edge  (i.e.  large  similarity  between  e*(d)  and  its  left  and  right  neighboring 
profiles),  <q(d)  should  have  a  shape  similar  to  a  Gaussian  function  with  a  large 
peak  and  a  small  standard  deviation.  We  define  the  edge  continuity  measure, 
Ki{d)  =  Pi  /<Ji(d)  in  which  Pc.  characterizes  the  peak  and  cq  is  the  standard 
deviation  of  a  Gaussian  function  fitted  to  <q(d).  For  each  point,  the  d  resulting  in 
the  maximum  continuity  measure  suggests  the  required  point  movement  along 
a  line  normal  to  the  boundary. 

We  define  un  to  be  the  vector  normal  to  the  boundary  at  the  boundary  point, 
dgE ,  to  be  the  distance  from  the  boundary  point  suggested  by  the  gray  level 
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similarity,  and  cIkus ,  and  cIke  to  be  the  distances  from  the  boundary  point 
computed  from  the  edge  continuity  measures  in  US  and  elastography  images, 
respectively.  Based  on  the  above  equations,  at  each  iteration,  fc,  the  following 
steps  are  performed  to  deform  the  current  shape  points,  xcurrent  into  the  next 
shape,  xnext • 

1.  Find  the  required  shape  deformation: 

dx  —  Xnext  Xcurrent  —  dfUn 

df  =  aidgE  +  ol^kus  +  ol^ke 

2.  Calculate  the  optimum  pose  parameters:  scaling,  translation  and 
rotation,  corresponding  to  dx,  apply  this  transformation  to  obtain 
T (xcurrent)-  Adjusting  the  pose  is  required  to  align  x current  to  be  as 
close  as  possible  to  xnext  before  adjusting  the  shape  [10]. 

3.  Calculate  db  =  PT(xnext  -  T(x  current )) 


a  =  [aq  ot2  0^3]  are  corresponding  weights.  Due  to  the  large  amount  of  noise 
in  US  images,  gray  level  similarity  matching  in  these  images  does  not  improve 
results  but  degrades  convergence,  and  therefore,  it  is  not  included.  Our  criteria 
for  convergence  is  when  94%  of  the  contour  points  have  a  dx  of  less  than  np/ 2. 
Fig.  1  illustrates  how  dgE  and  dxE  are  obtained  in  an  elastography  image. 

We  will  provide  a  comparison  of  segmentation  results  using  gray  level  similar¬ 
ity  from  elastography  images,  combined  edge  continuity  and  gray  level  similarity 
from  elastography  images  and  finally  edge  continuity  and  gray  level  similarity 
from  both  elastography  and  US  images. 


Fig.  1.  An  illustration  of  how  dgE  (top  point)  and  dxE  (bottom  point)  are  obtained 
in  an  elastography  image.  The  gray  level  similarity  measure  is  compared  to  that  of  the 
model  for  each  corresponding  point.  The  edge  continuity  measure  is  maximized  over 
the  line  normal  to  the  boundary.  For  clarity,  these  measures  are  shown  on  two  different 
points,  whereas  they  are  calculated  for  single  points. 


80 


S.S.  Mahdavi  et  al. 


3  Results 

To  evaluate  our  2D  segmentation  results  we  measure  the  mean  absolute  dis¬ 
tance  (MAD)  and  maximum  distance  (MaxD)  between  2D  automatic  and  man¬ 
ual  contours.  Table  1  provides  the  mean  and  standard  deviations  of  MAD  and 
MaxD  between  107  manual  and  automatic  contours  selected  from  7  patients. 
The  results  are  presented  separately  for  elastography  gray  level  similarity  only 
(aa=[l  0  0]),  elastography  gray  level  similarity  and  edge  continuity  (a^=[0.5 
0  0.5]),  and  elastography  gray  level  similarity  and  edge  continuity  plus  US  edge 
continuity  (<ac=[0.5  0.25  0.25]).  Fig.  2  shows  an  example  of  segmentation  results 
for  the  three  sets  of  weight  parameters. 

The  most  accurate  segmentation  results  were  acquired  when  ac=  [0.5  0.25  0.25] . 
By  using  this  selection  of  weight  parameters,  deformation  is  mainly  guided  by 
the  coarser  elastography  images  but  also  refined  by  the  finer  US  images.  It  is 
specifically  seen  in  the  posterior  region  of  the  prostate,  where  elastography  im¬ 
age  quality  is  low,  that  the  addition  of  edge  continuity  in  US  images  improves 
segmentation  results.  This  can  be  observed  in  Fig.  2.  The  choice  of  ac  also  results 
in  the  convergence  of  the  algorithm  in  an  average  of  22  iterations  vs.  37  and  98 
iterations  for  a ^  and  aa.  The  maximum  number  of  iterations  was  set  to  50  for 
and  ac  and  100  for  aa.  In  the  case  of  <aa,  95%  of  the  cases  did  not  converge 
within  100  iterations. 


Fig.  2.  2D  automatic  segmentation  results  using  elastography  gray  level  similarity 
(yellow  dashed  line),  elastography  edge  continuity  and  gray  level  similarity  (blue  line), 
and  elastography  gray  level  similarity  and  edge  continuity  plus  US  edge  continuity  (red 
line)  on  US  (left)  and  elastography  (right)  images.  The  manual  contour  is  shown  as  a 
blue/ white  dotted  line. 


4  Discussion 

By  visual  inspection  of  the  automatic  segmentation  results,  we  observed  that 
most  of  the  2D  segmentation  error  of  the  mid-gland  slice  was  in  the  anterior 
and  posterior  regions.  In  elastography  images,  the  blood  vessels  which  lie  on  the 
anterior  of  the  prostate  appear  as  stiff  tissue  and  are  not  always  distinguishable 
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Table  1.  Comparison  of  2D  manual  and  automatic  segmentation  showing  the  Mean 
Absolute  distance  (MAD)  and  Maximum  Distance  (MaxD  -  positive  sign  meaning 
larger  automatic  contour)  between  manual  and  automatic  prostate  contours,  the  num¬ 
ber  of  iterations  and  duration  of  the  algorithm.  K :  edge  continuity  measure,  dg:  gray 
level  similarity  measure. 


dg  in  elast. 

dg  and  K  in  elast. 

dg  and  K  in  elast. 
and  K  in  US 

MAD  (mm) 
MaxD  (mm) 
no.  of  iter, 
duration  (s) 

3.4  ±  1.8 

1.0  ±9.8 

98  =b  13 

10.7  zb  1.4 

3.4  zb  1.7 
-6.5  zb  7.0 

37  ±  17 

17.3  zb  0.8 

2.6  zb  0.9 
-4.9  dz  4.7 

22  ±  15 

17.9  zb  12.2 

from  the  prostate  itself.  In  such  cases,  the  automatic  contour  extends  beyond 
the  actual  boundary.  In  the  posterior,  due  to  the  relatively  low  contrast  in  this 
region,  the  automatic  contour  converges  to  the  darker  tissue  inside  the  prostate. 
By  including  edge  continuity  data  from  US  images  this  problem  has  been  par¬ 
tially  resolved.  We  attribute  this  low  contrast  to  mainly  the  slippage  between 
the  protective  sheath  on  the  probe  and  the  surface  of  the  rectum  during  elas- 
tography  data  collection.  By  replacing  the  ID  axial  strain  computation  with 
2D  axial/lateral  and  by  increasing  the  resolution  of  elastography  imaging  such 
problems  can  be  subsided. 

Our  current  elastography  data  acquisition  system  has  the  benefit  of  collecting 
inherently  registered  elastography  and  US  data.  Currently  the  TRUS  rotation 
range  is  within  ±50°  which  may  result  in  missing  data  in  the  mid-lateral  regions 
of  large  prostates.  Also,  the  quality  of  the  B-mode  US  images  acquired  along 
with  the  elastography  data  is  affected  by  the  computational  limitations  of  the 
real-time  data  acquisition  system  and  the  ultrasound  machine.  We  are  currently 
working  on  resolving  these  problems  to  improve  segmentation  results. 

The  proposed  2D  prostate  segmentation  method  using  fused  elastography  and 
US  image  information  can  be  extended  into  3D  by  modifying  the  method  that 
we  proposed  in  [12]  which  was  based  on  fitting  an  a  priori  shape  to  a  set  of 
parallel  transverse  ultrasound  images.  The  algorithm  was  initialized  by  the  user 
manually  selecting  some  initial  boundary  points  on  the  mid-slice.  These  points 
were  used  to  un-warp  and  un-taper  all  images  resulting  in  a  set  of  elliptical 
prostate  shapes.  With  the  aid  of  the  Interacting  Multiple  Model  Probabilistic 
Data  Association  (IMMPDA)  [13]  edge  detector  and  ellipse  fitting,  a  tapered 
ellipsoid  was  fitted  to  all  contours  which  was  then  sliced  at  image  depths.  The 
resulting  2D  contours  were  reversely  tapered  and  warped  to  match  the  initial 
images.  We  showed  [12]  that  the  method  is  fast,  and  produces  smooth  contours 
that  are  in  agreement  with  the  br  achy  therapy  requirements.  However,  the  need 
for  manual  initialization  limits  its  use  for  real-time  applications  and  makes  it 
user  dependent.  Also,  the  poor  visibility  of  the  boundary  at  the  base  and  apex 
complicates  segmentation  of  these  regions. 
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automatic 


Fig.  3.  An  example  of  3D  surface  of  the  prostate  created  manually  (red)  and  by  the 
automatic  algorithm  (blue) 


To  resolve  these  issues  we  replace  the  manual  initialization  of  the  3D  semi¬ 
automatic  segmentation  with  the  described  2D  segmentation.  Additionally,  we 
employ  elastography  along  with  US  images  for  propagating  the  mid-gland  seg¬ 
mentation  to  the  rest  of  the  images.  This  is  done  by  including  an  additional 
IMMPDA  edge  detection  on  the  coarser  elastography  images  to  guide  the  edge 
detection  on  the  finer  US  images.  The  described  framework  for  automatic  3D 
prostate  segmentation  was  applied  to  one  patient  data  set.  Fig.  3  shows  3D 
surfaces  created  from  automatic  (blue)  and  manual  (red)  segmentation  of  the 
prostate  for  this  patient.  11  images  were  used  to  construct  this  surface  model. 
The  volume  of  the  manually  created  surface  is  33.9  ml,  the  volume  of  the  au¬ 
tomatically  created  surface  is  34.4  ml,  and  the  volume  of  the  non-overlapping 
region  between  the  two  surfaces  is  4.9  ml.  A  thorough  clinical  study  is  required 
to  evaluate  this  3D  segmentation  framework. 

5  Conclusions 

In  this  paper  we  outlined  a  novel  2D  method  of  prostate  segmentation  that 
combines  ultrasound  elastography  imaging  with  B-mode  data.  This  is  the  first 
instance  of  using  such  a  combination  for  prostate  segmentation  and  reinforces 
efforts  to  improve  US  segmentation  outcomes  using  elastography  data  [14].  With 
the  fusion  of  information  from  elastic  properties  of  tissue  provided  by  elastogra¬ 
phy  with  the  acoustic  properties  of  tissue  provided  by  B-mode  we  developed  an 
automatic  and  accurate  segmentation  of  the  prostate  which  gives  good  results  in 
2D.  The  automatically  generated  2D  contours  can  be  used  to  initialize  the  mid¬ 
slice  for  3D  segmentation  and  remove  user  variability.  Additionally,  the  method 
can  be  utilized  to  register  pre  and  intra-operative  prostate  images  and  has  the 
potential  of  improving  intra-operative  dosimetry. 
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ULTRASOUND-BASED  TECHNIQUES  FOR  ENHANCING  DIAGNOSTIC  AND  THERAPEUTIC 
PROSTATE  INTERVENTIONS 
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Sahebjavaher,  Anthony  Koupparis,  Silvia  D.  Chang,  Edward  C.  Jones,  Robert  Rohling,  Piotr  Kozlowski, 
Christopher  Nguan,  and  S.  Larry  Goldenberg 
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Background  and  Objectives:  In  virtually  all  diagnostic  and  treatment  interventions  related  to  the  prostate,  new 
imaging  tools  that  can  combine  accurate  visualization  of  the  prostate  region  and  the  cancerous  tumors  could 
reduce  the  rate  of  complications  and  have  an  enormous  impact.  We  have  developed  ultrasound-based  imaging 
techniques  that  have  potential  in  improving  the  visualization  of  prostate  gland  and  detection  of  cancer.  These  are 
ultrasound  vibro-elastography  (VE)  for  visco-elastic  imaging  of  the  prostate  and  RF  time  series  analysis  for 
tissue  typing.  Within  our  2009  proposal  to  Department  of  Defense  Prostate  Cancer  Research  Program,  we  have 
proposed  to  use  these  methods  for  real-time  visualization  and  detection  of  the  boundaries  of  the  prostate  gland 
and  to  enhance  the  prostatectomy  and  brachytherapy  procedures  by  detecting  cancer  and  the  extent  of  extra- 
capsular  extension  of  prostate  cancer. 

Brief  Description  of  Methodologies:  An  LDR  brachytherapy  stepper  was  modified  to  enable  the  collection  of 
radio  frequency  (RF)  data  volumes  for  VE  and  time  series  imaging.  After  acquisition  of  the  institutional  ethics 
approval  and  informed  consent,  patients  undergoing  radical  prostatectomy  at  the  Vancouver  General  Hospital 
are  registered  into  this  study  and  go  through  pre-operative  magnetic  resonance  imaging  (MRI)  and  intra¬ 
operative  ultrasound  imaging.  The  ultrasound  RF  data  are  analyzed  offline  to  acquire  VE  images  and  RF  time 
series  feature  maps.  The  extracted  prostate  specimens  are  subjected  to  whole  mount  histopathology  analysis  to 
acquire  gold  standard  cancer  maps.  Both  VE  and  pathology  images  are  registered  to  MRI  images,  which  act  as  a 
link  between  VE  and  pathology  data  sets.  The  registration  approach  is  intensity-based  with  mutual  information 
as  the  similarity  measure. 

Results  to  Date:  Based  on  data  acquired  from  12  patients,  VE  images  improve  the  visualization  of  the  prostate 
gland.  VE  images  are  superior  to  B-mode  images  in  terms  of  contrast  (with  an  approximately  sixfold 
improvement  in  contrast-to-noise  ratio)  and  edge  strength  (with  an  approximately  twofold  improvement  in 
gradient  in  the  direction  normal  to  the  edge).  In  terms  of  cancer  detection,  we  have  received  three  sets  of 
histopathology  data,  which  show  that  prostate  tumors  appear  as  dark  areas  on  VE  images  due  to  their  increased 
stiffness. 

Conclusions:  VE  imaging  significantly  enhances  the  visualization  of  the  prostate  gland.  In  terms  of  cancer 
detection,  our  results  are  preliminary,  and  we  expect  to  increase  the  specificity  of  cancer  detection  by  the 
addition  of  the  RF  time  series  analysis  based  on  supervised  training,  and  combining  it  with  visco-elastic  features 
of  tissue  provided  by  VE. 

Impact  Statement:  If  successful,  the  combination  of  VE  and  RF  time  series  analysis  will  drastically  improve 
image -based  treatment  guidance.  In  particular,  surgical  resection  planes  could  be  defined  to  avoid  damaging 
critical  anatomical  structures. 

This  work  was  supported  by  the  U.S.  Army  Medical  Research  and  Materiel  Command  under  W81XWH- 10- 1-0201  and  the 
Natural  Sciences  and  Engineering  Research  Council  of  Canada. 
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Purpose:  With  an  estimated  240,890  cases  to  be  diagnosed  in  2011,  prostate  cancer  remains  the  most  common 
malignancy  (other  than  skin)  among  the  men  and  is  projected  to  cause  33,270  deaths.  A  non-invasive  imaging 
technique  that  can  reliably  characterize  and  grade  prostate  cancer  has  the  potential  to  improve  the  delivery  of 
prostate  interventions  such  as  biopsy,  prostatectomy  and  brachytherapy  by  providing  the  local  extent  of  the  disease 
and  facilitating  focal  therapy.  This  can  result  in  targeted  biopsies,  more  accurate  positive  margins  in  prostatectomy, 
and  enhanced  dynamic  dosimetry  in  brachytherapy.  We  have  previously  developed  cancer  detection  and 
characterization  techniques  based  on  both  multiparametric  MRI  [1]  and  ultrasound-based  methods  [2].  The  goal  of 
this  work  is  to  leverage  our  previous  efforts  and  experience  in  the  two  areas  and  create  the  translational  technology 
to  perform  MR-guided  interventions  assisted  by  tissue  typing  information  from  registered  MRI  and  ultrasound. 

Methods:  The  recent  launch  of  the  Advanced  Multimodality  Image  Guided  Operating  (AMIGO)  suite  at  BWH 
provides  us  with  a  unique  opportunity  to  develop  and  validate  the  enabling  technology  to  combine  the  state  of  the  art 
methods  in  ultrasound  and  MRI  based  tissue  typing  techniques.  The  AMIGO  suite  features  a  3T  MRI  wide  bore  (70  cm) 
Siemens  Verio  scanner  streaming  data  through  an  OpenIGT  link  to  3D  Sheer,  and  a  Pro  Focus  ultrasound  system  (BK 
Medical  Systems)  with  the  capability  of  streaming  both  the  B-mode  images  and  raw  RF  ultrasound  signals  to  an 
external  PC  through  a  camera  link  and  custom  software  under  development. 

The  access  to  ultrasound  RF  data  gives  us  the  ability  to  extract  tissue  typing  features  from  the  spectral 
analysis  of  the  RF  data  [2,3],  and  work  towards  developing  ultrasound  elastography  to  compute  tissue  elasticity.  MRI 
and  ultrasound  will  be  registered  in  3D  sheer  environment  and  tissue  typing  features  from  ultrasound  (texture 
features,  RF  spectral  features,  elasticity)  and  MRI  (DCE  and  DTI  features)  will  be  combined  for  tissue  classification.  An 
MR  compatible  and  US  transparent  adjustable  sleeve  template  assembly  is  being  developed  to  house  the  transrectal 
ultrasound  probe  and  the  MRI  coil,  interchangeably,  during  the  procedures.  This  will  ensure  similar  configurations  of 
the  prostate  and  the  rectal  wall  during  MRI  scan  and  the  ultrasound  scan  performed  using  a  non-MR  compatible 
transducer  after  the  MRI  magnet  is  moved  out  of  the  central  part  of  the  AMIGO  suite. 

Previous  experience:  Currently  in  BWH,  brachytherapy  procedures  are  performed  with  dynamic 
intraoperative  dosimetry  and  robotic  delivery.  MRI  is  acquired  pre-operatively  in  the  assessment  stage.  However, 
images  are  not  available  during  the  implant  procedure.  The  addition  of  MR-ultrasound  fusion  to  the  procedure  will  be 
the  first  step  in  the  new  initiative.  Transition  to  AMIGO  will  follow  and  will  take  advantage  of  the  MRI-compatible 
robotic  technology  developed  in  our  group  [4].  In  the  past  we  have  reported  on  the  BWH  clinical  experience  with  3T 
MRI-guided  transperineal  prostate  biopsy  [5],  and  have  now  completed  the  first  case  of  that  type  in  AMIGO. 

Conclusions:  The  AMIGO  suite  affords  us  the  opportunity  to  develop  and  quantitatively  validate  the  effects  of 
registered  MRI  and  ultrasound  tissue  typing.  The  final  product  will  be  translated  to  the  conventional  TRUS-guided 
interventional  environment  by  using  3D  ultrasound  as  the  intraoperative  modality  and  registration  to  pre-operative 
MRI.  This  will  enable  wider  accessibility  of  the  developed  technology. 
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